@thailandhosting/object-storage 0.0.0-stage → 1.1.0

package/README.md

@@ -1,3 +1,203 @@
-# Temporary Holding Version
+# ThailandHosting Object Storage — Node.js SDK
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+TypeScript/JavaScript client for ThailandHosting Object Storage: buckets,
+objects, large files and share links over the ThailandHosting API.
+
+```
+npm install @thailandhosting/object-storage
+```
+
+Node.js 18.17+. ESM with TypeScript types. No runtime dependencies: it uses
+Node's built-in `fetch` (keep-alive connection pooling) and streams.
+
+## Quick start
+
+```ts
+import { Client, NotFoundError } from '@thailandhosting/object-storage';
+
+const client = new Client({
+  endpoint: 'https://objects.thailandhosting.com/th-bkk-1',
+  accessKeyId: process.env.TH_ACCESS_KEY_ID!,
+  secretAccessKey: process.env.TH_SECRET_ACCESS_KEY!,
+});
+const bucket = client.bucket('my-bucket');
+
+// Any size: large files go up as parallel parts.
+await bucket.uploadFile('backups/db.dump', '/var/backups/db.dump');
+
+// Large objects come down as parallel ranges.
+await bucket.downloadFile('backups/db.dump', '/tmp/db.dump');
+
+// Small objects
+await bucket.put('notes/hello.txt', 'สวัสดี', { metadata: { author: 'me' } });
+console.log(await bucket.getText('notes/hello.txt'));
+
+// Stream an upload of unknown length (an HTTP request, a pipe, a generated export)
+await bucket.upload('exports/report.csv', someReadable);
+
+// Stream a download
+const dl = await bucket.get('videos/intro.mp4');
+dl.toNodeStream().pipe(res);
+
+// Every object under a prefix
+for await (const obj of bucket.objects('backups/')) console.log(obj.key, obj.size);
+
+// A link that works without a key for one hour
+console.log((await bucket.createLink('backups/db.dump', { expiresIn: 3600 })).url);
+
+try {
+  await bucket.head('missing');
+} catch (e) {
+  if (e instanceof NotFoundError) console.log(e.code); // object_not_found
+}
+```
+
+Create access keys in the Console (Object Storage → Access keys). Reuse one
+`Client`: it keeps connections open. Every call takes an optional
+`AbortSignal`.
+
+## Authentication
+
+Every request is signed with HTTP Message Signatures (RFC 9421,
+HMAC-SHA256 keyed with the secret access key). The secret itself is never
+sent: the request carries `Signature-Input` (covering the method, host, path,
+query and, with a body, its `Content-Digest`) and `Signature`. Each attempt is
+signed afresh, so the machine's clock must be within 5 minutes of the
+service's. A client without a key sends no signature.
+
+`signRequest` and `contentDigest` are exported for signing requests made
+with your own HTTP client.
+
+## Using a service account
+
+A program can use a service account's short-lived token instead of an access
+key. Get it with the identity SDK and pass `tokenProvider` in place of the key:
+
+```ts
+import { defaultCredential } from '@thailandhosting/identity';
+
+const cred = defaultCredential();
+const client = new Client({
+  endpoint: 'https://objects.thailandhosting.com/th-bkk-1',
+  tokenProvider: () => cred.getToken(),
+});
+```
+
+The account needs a grant on the bucket, set in the Console. Creating and
+deleting buckets and changing public access take an access key allowed to
+manage buckets.
+
+`tokenProvider` is called before every attempt, so a refreshed token is picked
+up. Give a key or a token provider, not both. If it fails, nothing is sent or
+retried: the call rejects with `TokenError`. Large uploads and downloads work
+as with a key. `createLink` needs an access key, because a link is signed with
+the key's secret, and rejects with a token. `publicUrl` works.
+
+## Conditional writes
+
+```ts
+import { PreconditionFailedError } from '@thailandhosting/object-storage';
+
+// Create only if the key is free (also on uploadFile / upload, where the
+// condition is checked when a large upload's parts are joined).
+await bucket.put('locks/job-42', owner, { ifNoneMatch: '*' });
+
+// Replace only the version you read; delete only if unchanged.
+const info = await bucket.head('config.json');
+try {
+  await bucket.put('config.json', next, { ifMatch: info.etag });
+} catch (e) {
+  if (e instanceof PreconditionFailedError) console.log('changed meanwhile'); // code precondition_failed
+}
+await bucket.delete('config.json', { ifMatch: info.etag });
+
+// Reads: NotModifiedError when the cached copy is current.
+await bucket.get('logo.png', { ifNoneMatch: cachedETag });
+await bucket.head('logo.png', { ifModifiedSince: cachedAt });
+```
+
+The service checks `ifMatch` / `ifNoneMatch` atomically with the write. A
+failed condition is `PreconditionFailedError` (HTTP 412) and is never retried.
+
+## Public buckets and folders
+
+```ts
+// Anyone may read images/ (and nothing else) without a key; needs a key
+// allowed to manage buckets. [] makes the bucket private again.
+await bucket.setPublicAccess([{ prefix: 'images/' }]);
+console.log(bucket.publicUrl('images/logo.png'));
+// https://objects.thailandhosting.com/th-bkk-1/my-bucket/images/logo.png
+
+// A client without a key reads (and, where allowed, lists) public objects.
+const anon = new Client({ endpoint: 'https://objects.thailandhosting.com/th-bkk-1' });
+const logo = await anon.bucket('my-bucket').getBytes('images/logo.png');
+```
+
+What a key may do (buckets, folders, read / list / write / delete,
+managing buckets, an expiry date) is set on the key in the Console.
+
+## Performance
+
+| | |
+|---|---|
+| Connections | Node's `fetch` (undici): keep-alive pooling, no per-request handshake |
+| Uploads | `uploadFile` reads parts with positional reads and sends them in parallel; `upload` streams any `Readable` / `ReadableStream` / async iterable, holding at most `concurrency + 1` parts in memory |
+| Downloads | `downloadFile` fetches parallel ranges and writes each straight to its place in the file; `get` streams |
+| Small data | one request; a stream that turns out small is one request too |
+| Tuning | `partSize` (default 16 MiB), `concurrency` (default 8), `multipartThreshold` (default 32 MiB) |
+
+## Reliability
+
+- Requests that are safe to repeat are retried with exponential backoff and
+  jitter (`maxRetries`, default 4) on 429, 500, 502, 503, 504 and network
+  errors, waiting at least as long as `Retry-After` asks.
+- Every request body (each part, and JSON) carries a `Content-Digest`
+  (SHA-256); the service checks it as the bytes arrive and refuses a body
+  that changed on the way (`bad_digest`, not retried). Streams are cut into
+  parts in memory, so their digest is known before they are sent.
+- Every single upload and every part is checked against the MD5 the service
+  reports; damaged data throws `IntegrityError` and is never accepted.
+- An object uploaded in one request has a `Repr-Digest` (`info.digest`).
+  Whole-object downloads (`getBytes`, `getText`, `get` without a range,
+  `downloadFile` including its parallel ranges) are checked against it and
+  throw `IntegrityError` if the bytes differ. Multipart uploads have none.
+- A failed multipart upload is aborted, so its parts are not billed.
+- `downloadFile` writes to `path + '.download'` and renames it into place;
+  every range is pinned to the object's ETag, and a broken range resumes
+  where it stopped.
+
+## API
+
+| `Client` | |
+|---|---|
+| `listBuckets()`, `createBucket(name)`, `getBucket(name)`, `deleteBucket(name)` | buckets |
+| `bucket(name)` | a `Bucket` handle |
+
+| `Bucket` | |
+|---|---|
+| `put`, `upload`, `uploadFile` | upload (`contentType`, `metadata`, `cacheControl`, `contentDisposition`, `contentEncoding`, `ifMatch`, `ifNoneMatch`, `ifNotExists`) |
+| `get` (`offset`/`length`, `ifNoneMatch` / `ifModifiedSince` → `NotModifiedError`, `ifMatch` / `ifUnmodifiedSince`), `getBytes`, `getText`, `downloadFile` | download |
+| `head` (same conditions as `get`), `exists`, `delete` (`ifMatch`, `ifNoneMatch`), `deleteMany`, `copy` | object operations |
+| `list` (one page), `objects` (async iterator) | listing |
+| `createLink(key, { expiresIn, upload })` | a link that needs no key |
+| `publicAccess()`, `setPublicAccess(rules)`, `publicUrl(key)` | make the bucket or some folders readable by anyone, and their URLs |
+| `createMultipartUpload`, `resumeMultipartUpload` → `uploadPart`, `parts`, `complete` (`ifMatch`, `ifNoneMatch`), `abort` | do-it-yourself multipart |
+
+The service answers errors as Problem Details (RFC 9457,
+`application/problem+json`). They throw `ObjectStorageError` with `status`,
+`code` (for example `object_not_found`, `bucket_already_exists`, `busy`), the
+problem's title in `message`, and `type` / `instance`. `NotFoundError` is the
+404 case and `PreconditionFailedError` the 412 one.
+
+## Development
+
+```
+npm install
+npm test          # runs the TypeScript tests directly (Node 22.6+ type stripping)
+npm run build     # dist/ (ESM + .d.ts)
+```
+
+`npm test` uses an in-memory fake of the service. Set
+`TH_OBJECT_STORAGE_ENDPOINT`, `TH_ACCESS_KEY_ID` and `TH_SECRET_ACCESS_KEY`
+to also run a round trip against a real endpoint (it creates and deletes a
+bucket).

package/dist/client.d.ts

@@ -0,0 +1,265 @@
+import { Readable } from 'node:stream';
+export declare const VERSION = "1.0.0";
+export interface ClientOptions {
+    /** The region's endpoint, e.g. https://objects.bkk.thailandhosting.com */
+    endpoint: string;
+    /** With secretAccessKey; leave both out for an anonymous client that reads only what buckets made public.
+     *  Requests are signed with the secret (HTTP Message Signatures); the secret itself is never sent. */
+    accessKeyId?: string;
+    secretAccessKey?: string;
+    /** Instead of a key: gives a service account's short-lived token, sent as `Authorization: Bearer` with no
+     *  signature. Called before every attempt, retries included, so it should cache and refresh the token.
+     *  `createLink` needs an access key. */
+    tokenProvider?: () => Promise<string> | string;
+    /** Size of each upload part and download range (default 16 MiB, at least 5 MiB). */
+    partSize?: number;
+    /** Size above which transfers are split (default 2 × partSize). */
+    multipartThreshold?: number;
+    /** Parts or ranges in flight at once (default 8). */
+    concurrency?: number;
+    /** Retries of requests that are safe to repeat (default 4). */
+    maxRetries?: number;
+    /** Appended to the SDK's User-Agent. */
+    userAgent?: string;
+    /** A fetch implementation (default: the global fetch). */
+    fetch?: typeof fetch;
+}
+export interface BucketInfo {
+    name: string;
+    region: string;
+    createdAt?: Date;
+    /** What anyone may read; getBucket fills it, listBuckets does not. */
+    public?: PublicRule[];
+}
+/** Makes the objects under `prefix` ("" is the whole bucket) readable by anyone without a key; `list` also lets anyone list those keys. */
+export interface PublicRule {
+    prefix: string;
+    list?: boolean;
+}
+export interface ObjectInfo {
+    key: string;
+    size: number;
+    etag: string;
+    lastModified?: Date;
+    contentType?: string;
+    metadata: Record<string, string>;
+    /** The object's Repr-Digest (`sha-256=:<base64>:`), from get and head, when it was uploaded in
+     *  one request; full downloads are checked against it. Multipart uploads have none. */
+    digest?: string;
+}
+/** Conditions checked by the service atomically with a write or delete; a false one throws
+ *  PreconditionFailedError. An ETag (quotes optional), a comma-separated list of them, or `*`. */
+export interface WriteConditions {
+    /** Only if the object's ETag is one of these (`*`: only if it exists). */
+    ifMatch?: string;
+    /** Only if the object's ETag is none of these (`*`: only if the key is free). */
+    ifNoneMatch?: string;
+}
+export interface PutOptions extends WriteConditions {
+    contentType?: string;
+    cacheControl?: string;
+    contentDisposition?: string;
+    contentEncoding?: string;
+    /** Stored with the object, returned as X-Meta-<name> (2 KB in all). */
+    metadata?: Record<string, string>;
+    /** Fail with code `precondition_failed` if the key already holds an object (`ifNoneMatch: '*'`). */
+    ifNotExists?: boolean;
+    signal?: AbortSignal;
+}
+export interface HeadOptions {
+    /** Throw NotModifiedError when the object's ETag is one of these. */
+    ifNoneMatch?: string;
+    /** Throw PreconditionFailedError unless the object's ETag is one of these. */
+    ifMatch?: string;
+    /** Throw NotModifiedError unless the object changed after this. */
+    ifModifiedSince?: Date;
+    /** Throw PreconditionFailedError if the object changed after this. */
+    ifUnmodifiedSince?: Date;
+    signal?: AbortSignal;
+}
+export interface GetOptions extends HeadOptions {
+    /** Download `length` bytes from `offset` (length omitted: to the end). */
+    offset?: number;
+    length?: number;
+}
+export interface DeleteOptions extends WriteConditions {
+    signal?: AbortSignal;
+}
+export interface CompleteOptions extends WriteConditions {
+    signal?: AbortSignal;
+}
+export interface ListOptions {
+    prefix?: string;
+    /** Roll keys up into folders ("/" lists one level). */
+    delimiter?: string;
+    cursor?: string;
+    /** Page size, 1-1000 (default 1000). */
+    limit?: number;
+    signal?: AbortSignal;
+}
+export interface ListPage {
+    objects: ObjectInfo[];
+    folders: string[];
+    /** Continues the listing; undefined on the last page. */
+    nextCursor?: string;
+}
+export interface Part {
+    number: number;
+    etag: string;
+    size: number;
+}
+export interface Link {
+    url: string;
+    method: 'GET' | 'PUT';
+    expiresAt?: Date;
+}
+/** A download in progress. Read `body` (a web stream) or `toNodeStream()`. */
+export interface ObjectDownload {
+    info: ObjectInfo;
+    contentRange?: string;
+    body: ReadableStream<Uint8Array>;
+    toNodeStream(): Readable;
+}
+type Body = string | Uint8Array;
+interface RequestInit2 {
+    method: string;
+    path: string;
+    query?: Record<string, string | number | undefined>;
+    headers?: Record<string, string>;
+    body?: Uint8Array;
+    /** Retry a POST on answers that mean nothing was done (429, 503). */
+    retryPost?: boolean;
+    /** Statuses that are not errors (default: any 2xx). */
+    expect?: number[];
+    signal?: AbortSignal;
+}
+/** A client of one region's Object Storage endpoint. Reuse it: it keeps connections open. */
+export declare class Client {
+    readonly partSize: number;
+    readonly multipartThreshold: number;
+    readonly concurrency: number;
+    readonly maxRetries: number;
+    private readonly base;
+    /** @internal */ readonly origin: string;
+    private readonly keyId;
+    private readonly secret;
+    /** @internal */ readonly tokenProvider?: () => Promise<string> | string;
+    private readonly ua;
+    private readonly fetch;
+    constructor(opts: ClientOptions);
+    /** @internal Sends a request with retries; the caller reads or cancels the body. */
+    request(r: RequestInit2): Promise<Response>;
+    /** @internal */
+    json<T = any>(method: string, path: string, payload?: unknown, opts?: {
+        query?: RequestInit2['query'];
+        retryPost?: boolean;
+        signal?: AbortSignal;
+        headers?: Record<string, string>;
+    }): Promise<T>;
+    /** The buckets this access key can use. */
+    listBuckets(signal?: AbortSignal): Promise<BucketInfo[]>;
+    /** Create a bucket (3-63 characters of a-z, 0-9, dots and hyphens). */
+    createBucket(name: string, signal?: AbortSignal): Promise<BucketInfo>;
+    getBucket(name: string, signal?: AbortSignal): Promise<BucketInfo>;
+    /** Delete an empty bucket. */
+    deleteBucket(name: string, signal?: AbortSignal): Promise<void>;
+    /** A handle for one bucket's objects (no request is made). */
+    bucket(name: string): Bucket;
+    /** @internal */
+    partSizeFor(size: number): number;
+}
+/** The objects of one bucket. */
+export declare class Bucket {
+    readonly name: string;
+    /** @internal */ readonly c: Client;
+    /** @internal */ readonly path: string;
+    constructor(client: Client, name: string);
+    private obj;
+    /** Upload a string or bytes in one request. */
+    put(key: string, data: Body, opts?: PutOptions): Promise<ObjectInfo>;
+    /** Stream an object (or a range of it). `info.size` is the whole object's size. A whole-object
+     *  download with a Repr-Digest is checked as it is read: the body fails with IntegrityError at
+     *  its end if the bytes do not match. */
+    get(key: string, opts?: GetOptions): Promise<ObjectDownload>;
+    /** Download a whole object into memory, checked against its Repr-Digest when it has one. */
+    getBytes(key: string, signal?: AbortSignal): Promise<Uint8Array>;
+    /** Download a whole object as UTF-8 text. */
+    getText(key: string, signal?: AbortSignal): Promise<string>;
+    /** An object's info without its data. With `ifNoneMatch` / `ifModifiedSince` it throws
+     *  NotModifiedError when the object is unchanged. */
+    head(key: string, opts?: AbortSignal | HeadOptions): Promise<ObjectInfo>;
+    exists(key: string, signal?: AbortSignal): Promise<boolean>;
+    /** Delete an object; deleting a missing one succeeds (unless `ifMatch` wanted one). */
+    delete(key: string, opts?: AbortSignal | DeleteOptions): Promise<void>;
+    /** Delete keys in parallel. */
+    deleteMany(keys: string[], signal?: AbortSignal): Promise<void>;
+    /** Copy inside the service, without downloading. */
+    copy(srcKey: string, dstKey: string, opts?: {
+        sourceBucket?: string;
+        metadata?: Record<string, string>;
+        signal?: AbortSignal;
+    }): Promise<ObjectInfo>;
+    /** One page of objects (and folders, with a delimiter). */
+    list(opts?: ListOptions): Promise<ListPage>;
+    /** Every object under prefix: `for await (const obj of bucket.objects('photos/'))`. */
+    objects(prefix?: string, opts?: {
+        pageSize?: number;
+        signal?: AbortSignal;
+    }): AsyncGenerator<ObjectInfo>;
+    /** A URL that downloads (or, with `upload: true`, uploads) one object without a key. Needs a client with an access key. */
+    createLink(key: string, opts?: {
+        expiresIn?: number;
+        upload?: boolean;
+        signal?: AbortSignal;
+    }): Promise<Link>;
+    /** What anyone may read in the bucket. */
+    publicAccess(opts?: {
+        signal?: AbortSignal;
+    }): Promise<PublicRule[]>;
+    /**
+     * Replace what anyone may read: `[]` makes the bucket private, `[{ prefix: '' }]` opens all of it,
+     * `[{ prefix: 'images/' }]` one folder. Needs a key allowed to manage buckets; takes effect within
+     * about 15 seconds. Resolves to the rules as stored.
+     */
+    setPublicAccess(rules: PublicRule[], opts?: {
+        signal?: AbortSignal;
+    }): Promise<PublicRule[]>;
+    /** The address anyone can open `key` at once its folder (or the bucket) is public. No request is made. */
+    publicUrl(key: string): string;
+    /** Start a multipart upload (uploadFile and upload do this for you). */
+    createMultipartUpload(key: string, opts?: PutOptions): Promise<MultipartUpload>;
+    resumeMultipartUpload(key: string, uploadId: string): MultipartUpload;
+    /** Upload a local file. Large files go up as parts read from the file and
+     *  sent in parallel; a part is re-read if it must be sent again. `ifMatch` /
+     *  `ifNoneMatch` are checked when the parts are joined. */
+    uploadFile(key: string, path: string, opts?: PutOptions): Promise<ObjectInfo>;
+    /** Upload what a stream yields (a pipe, an HTTP request, a generated
+     *  export) without knowing its size. Parts are sent in parallel, holding at
+     *  most `concurrency + 1` in memory; every part (or a small stream, sent in
+     *  one request) is buffered, so its Content-Digest is known before it is sent. */
+    upload(key: string, source: AsyncIterable<Uint8Array> | ReadableStream<Uint8Array>, opts?: PutOptions): Promise<ObjectInfo>;
+    /** Save an object to `path`. Large objects come down as parallel ranges
+     *  pinned to the object's ETag, written straight to their place in the
+     *  file; the data goes to `path + '.download'` and is renamed into place
+     *  only once it matches the object's Repr-Digest (when it has one). */
+    downloadFile(key: string, path: string, signal?: AbortSignal): Promise<ObjectInfo>;
+    private downloadRange;
+}
+/** A large upload in progress: send parts (any order, in parallel), then complete() or abort(). */
+export declare class MultipartUpload {
+    readonly bucket: Bucket;
+    readonly key: string;
+    readonly uploadId: string;
+    constructor(bucket: Bucket, key: string, uploadId: string);
+    private get path();
+    /** Send part `n` (1-10000); every part but the last must be at least 5 MiB. */
+    uploadPart(n: number, data: Body, signal?: AbortSignal): Promise<Part>;
+    parts(signal?: AbortSignal): Promise<Part[]>;
+    /** Join the parts into the object; without `parts`, every part received, in order.
+     *  `ifMatch` / `ifNoneMatch` are checked against the key atomically with the join. */
+    complete(parts?: Part[], opts?: AbortSignal | CompleteOptions): Promise<ObjectInfo>;
+    abort(signal?: AbortSignal): Promise<void>;
+    /** @internal */
+    abortQuietly(): Promise<void>;
+}
+export {};