@filefreak.io/sdk 0.0.0-stage → 0.2.0

package/README.md

@@ -1,3 +1,190 @@
-# Temporary Holding Version
+# @filefreak.io/sdk
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The FileFreak developer API, from TypeScript or JavaScript, in Node 20 and
+later and in browsers. No dependencies.
+
+Every endpoint is a function, typed from the same table the API is built
+from. Around them: uploads sent straight to storage in parts and carried on
+after an interruption, downloads fetched in ranges, listings and changes read
+to the end, files under Shard Protection read through their maps, what a
+share link holds saved with no key, and requests sent again only when that
+cannot make anything twice. The reference is at
+https://filefreak.io/developers/reference.
+
+## On a server
+
+```ts
+import { FileFreak } from "@filefreak.io/sdk";
+import { fromPath, toPath } from "@filefreak.io/sdk/node";
+
+const ff = new FileFreak({ apiKey: process.env.FILEFREAK_KEY });
+
+const { item: folder } = await ff.folders.create({ name: "uploads", ifExists: "use" });
+const file = await ff.upload(await fromPath("film.mp4"), {
+  parentId: folder.id,
+  onProgress: (sent, total) => console.log(`${Math.round((sent / total) * 100)}%`),
+});
+
+await ff.download(file.id, await toPath("copy.mp4"));
+const bytes = await ff.bytes(file.id); // for a file small enough to hold in memory
+```
+
+`fromPath` reads the file a part at a time as it is sent, so a file larger
+than memory is fine. `toPath` writes beside the file and gives it its name
+only once every byte is there: a download that fails leaves nothing behind.
+A file under Shard Protection has no address, and `download` and `bytes`
+read it through its map instead, each fragment checked as it arrives.
+
+Keep keys on a server. A key sent from a web page is refused.
+
+## In a web page
+
+A page is given a token by your server instead. Your server makes it with its
+key:
+
+```ts
+// On your server, behind your own sign-in.
+const { token } = await ff.tokens.create({
+  scopes: ["files:read", "files:write"],
+  folderId: customerFolderId,
+  expiresInSeconds: 900,
+});
+```
+
+and the page uses it as it would a key. Given a function rather than a
+string, the SDK asks your server for a new token when the last has run out:
+
+```ts
+import { FileFreak } from "@filefreak.io/sdk";
+
+const ff = new FileFreak({
+  token: async () => (await fetch("/my-server/filefreak-token")).text(),
+});
+
+const input = document.querySelector<HTMLInputElement>("input[type=file]")!;
+const file = await ff.upload(input.files![0]!); // into the token's folder
+```
+
+A token may do what its key may, or less: the scopes asked for, inside the
+folder asked for, for a minute to an hour. It is checked against its key on
+every request, so revoking the key stops its tokens at once. A token cannot
+make another.
+
+## Saving from a link
+
+`get` saves what a share link or a public page holds. It needs no key, and
+sends nothing that says who is asking: no key, and from a browser no
+cookies. A file goes into one sink; a folder needs somewhere for each of its
+files, which `toFolder` makes on disk.
+
+```ts
+import { get } from "@filefreak.io/sdk";
+import { toFolder, toPath } from "@filefreak.io/sdk/node";
+
+await get("https://filefreak.link/share/...", await toPath("report.pdf"));
+
+const got = await get("https://filefreak.link/share/...", toFolder("downloads"), {
+  password: "open sesame", // for a link that has one
+  onProgress: (written, total) => console.log(`${written} of ${total} bytes`),
+});
+console.log(`${got.files.length} files, ${got.sizeBytes} bytes`);
+```
+
+It takes a share link, a public file's or folder's page, and a file's direct
+address. A folder comes whole: each file goes to the sink the function makes
+for it, given its name and the folders it is in. Each file saved is one of
+the link's downloads, as on its page, and a link with fewer left than there
+are files saves nothing (`not_enough_downloads`). A file under Shard
+Protection is read through the link's map, and counted once.
+
+Every name a link gives is its sharer's, and can be anything: `../../.bashrc`,
+or one that reads backwards. `toFolder` makes each one safe with `safeName`
+before writing, and never writes outside the folder it is given. A program
+that names files from a link itself should use `safeName` too.
+
+## Every endpoint
+
+The functions are grouped as the reference groups the endpoints:
+`ff.files`, `ff.folders`, `ff.tree`, `ff.changes`, `ff.uploads`, `ff.usage`,
+`ff.me`, `ff.tokens` and `ff.shares` (share links, which only a device signed
+in with `filefreak login` may make). Ids in the address come first, then the body or
+the query, then options:
+
+```ts
+const { items, nextCursor } = await ff.files.list({ parent: folder.id, limit: 50 });
+await ff.files.update(file.id, { name: "renamed.mp4" });
+await ff.files.delete(file.id, { permanent: true });
+```
+
+`ff.call("files.get", { params: { id } })` reaches any endpoint by its name.
+
+## Reading to the end
+
+```ts
+for await (const item of ff.files.each({ parent: folder.id })) console.log(item.name);
+
+// Everything under a folder, then only what changes.
+const walk = ff.tree.walk({ parent: folder.id });
+for await (const item of walk) keep(item);
+for await (const answer of ff.changes.follow(walk.changesCursor)) {
+  for (const change of answer.changes) apply(change);
+  save(answer.cursor); // to carry on from after a restart
+}
+```
+
+`follow` waits on the API for the next change, a minute at a time, and goes
+on until the loop is left or its `signal` is aborted.
+
+A copy that also sends its own changes back names the file as it read it,
+so that it never writes over, or deletes, a change it has not seen. Either
+is refused with 409 `revision_changed` if the file has moved on:
+
+```ts
+await ff.upload(source, { name: file.name, parentId: file.parentId, onConflict: "replace", replacing: { id: file.id, revision: file.revision } });
+await ff.files.delete(file.id, { ifRevision: file.revision });
+```
+
+## When an upload stops
+
+What reached storage is kept until the upload expires. Keep the upload's id
+from `onBegun`, and carry on with the same file: only the parts storage does
+not have are sent.
+
+```ts
+import { UploadStopped } from "@filefreak.io/sdk";
+
+try {
+  await ff.upload(source, { name: "big.iso", onBegun: (u) => remember(u.id) });
+} catch (e) {
+  if (e instanceof UploadStopped) await ff.upload(source, { resume: e.uploadId });
+  else throw e;
+}
+```
+
+## When something is refused
+
+A refusal is a `FileFreakError`, with the API's `status`, a `code` that does
+not change and is what to branch on, and a `message` for a person.
+
+```ts
+import { FileFreakError } from "@filefreak.io/sdk";
+
+try {
+  await ff.folders.create({ name: "reports" });
+} catch (e) {
+  if (e instanceof FileFreakError && e.code === "name_taken") {
+    // already there
+  } else throw e;
+}
+```
+
+The SDK sends a request again only when that cannot make anything twice: a
+read; a request with an Idempotency-Key, which it makes for every endpoint
+that honours one unless you give your own (`{ idempotencyKey }` in the
+options); completing an upload or making a token; or an answer saying nothing
+was done, such as 429. While the service is waking up after a quiet spell it
+waits, for up to a few minutes. `Retry-After` is honoured.
+
+## Licence
+
+Apache-2.0.

package/dist/client.d.ts

@@ -0,0 +1,78 @@
+import { bind, type ChangesListAnswer, type FilesListQuery, type Input, type Item, type OperationName, type TreeListQuery } from "./generated.js";
+import { type Sink } from "./ranges.js";
+import { type CallOptions, type ClientOptions } from "./transport.js";
+import { type Source, type UploadOptions } from "./uploads.js";
+type Api = ReturnType<typeof bind>;
+export interface DownloadOptions {
+    /** Stops the download, and aborts the sink. */
+    signal?: AbortSignal;
+    /** Called as bytes are written, with how many have been and the file's size. */
+    onProgress?: (writtenBytes: number, totalBytes: number) => void;
+    /** At most this many ranges at once, up to six. */
+    ranges?: number;
+}
+export interface FollowOptions extends CallOptions {
+    /** Seconds each request waits for a change before asking again, 0 to 200. */
+    wait?: number;
+    /** At most this many changes in one answer. */
+    limit?: number;
+}
+/**
+ * Everything under a folder, read to the end a page at a time. Once the first
+ * page has arrived, changesCursor holds the place in the changes from before
+ * the walk began: follow the changes from there after the last item, and
+ * nothing that changed during the walk is missed.
+ */
+export declare class Walk implements AsyncIterable<Item> {
+    private readonly list;
+    private readonly query;
+    private readonly options;
+    changesCursor: string | null;
+    constructor(list: Api["tree"]["list"], query: Omit<TreeListQuery, "cursor">, options: CallOptions | undefined);
+    [Symbol.asyncIterator](): AsyncGenerator<Item>;
+}
+export declare class FileFreak {
+    private readonly transport;
+    readonly files: Api["files"] & {
+        /** Everything in a folder, read to the end a page at a time. */
+        each(query?: Omit<FilesListQuery, "cursor">, options?: CallOptions): AsyncIterable<Item>;
+    };
+    readonly folders: Api["folders"];
+    readonly tree: Api["tree"] & {
+        /** Everything under a folder, read to the end. See Walk. */
+        walk(query?: Omit<TreeListQuery, "cursor">, options?: CallOptions): Walk;
+    };
+    readonly changes: Api["changes"] & {
+        /**
+         * The changes after `cursor` (or after now, without one), for as long as
+         * the loop reading them goes on: each answer that has any, oldest first.
+         * Apply its changes, then keep its cursor, to carry on from after a
+         * restart. Stopped by options.signal, or by leaving the loop.
+         */
+        follow(cursor?: string | null, options?: FollowOptions): AsyncIterable<ChangesListAnswer>;
+    };
+    readonly uploads: Api["uploads"];
+    readonly usage: Api["usage"];
+    readonly tokens: Api["tokens"];
+    constructor(options: ClientOptions);
+    /** Any endpoint, by its operation name. The functions above are this, typed. */
+    call(operation: OperationName, input?: Input, options?: CallOptions): Promise<unknown>;
+    /**
+     * Send a file straight to storage in parts, and make it a file. The API is
+     * told what is coming and when it has all arrived; the bytes go from here
+     * to storage. See UploadOptions, and UploadStopped for carrying on.
+     */
+    upload(source: Source, options?: UploadOptions): Promise<Item>;
+    /**
+     * Fetch a file straight from storage into `sink`: as up to six ranges at
+     * once for a large one. One download address is asked for, so the file is
+     * counted toward the month's downloads once. A file under Shard Protection
+     * has no address, and is read through its map instead (files.shards), each
+     * fragment checked as it arrives. Resolves to the file.
+     */
+    download(id: string, sink: Sink, options?: DownloadOptions): Promise<Item>;
+    /** A file's bytes, in memory: for a file small enough to hold. */
+    bytes(id: string, options?: DownloadOptions): Promise<Uint8Array<ArrayBuffer>>;
+    private fetchInto;
+}
+export {};

package/dist/client.js

@@ -0,0 +1,168 @@
+// The client: every endpoint of the developer API as a function, made from
+// the API's own table (generated.ts), and the pieces a program would
+// otherwise write around them: sending a file in parts, fetching one in
+// ranges, and reading a listing or the changes to the end.
+import { bind, OPERATIONS, } from "./generated.js";
+import { fetchRanges } from "./ranges.js";
+import { asItIsKept, readProtected } from "./reader.js";
+import { FileFreakError, sleep, Transport } from "./transport.js";
+import { upload } from "./uploads.js";
+/** How long a follow of the changes waits for one before asking again, unless told. */
+const FOLLOW_WAIT_SECONDS = 60;
+/** A follow asks at most once a second while nothing is changing. */
+const MIN_ASK_MS = 1000;
+/**
+ * Everything under a folder, read to the end a page at a time. Once the first
+ * page has arrived, changesCursor holds the place in the changes from before
+ * the walk began: follow the changes from there after the last item, and
+ * nothing that changed during the walk is missed.
+ */
+export class Walk {
+    list;
+    query;
+    options;
+    changesCursor = null;
+    constructor(list, query, options) {
+        this.list = list;
+        this.query = query;
+        this.options = options;
+    }
+    async *[Symbol.asyncIterator]() {
+        let cursor = null;
+        let first = true;
+        do {
+            const page = await this.list(cursor ? { ...this.query, cursor } : this.query, this.options);
+            if (first)
+                this.changesCursor = page.changesCursor;
+            first = false;
+            yield* page.items;
+            cursor = page.nextCursor;
+        } while (cursor);
+    }
+}
+export class FileFreak {
+    transport;
+    files;
+    folders;
+    tree;
+    changes;
+    uploads;
+    usage;
+    tokens;
+    constructor(options) {
+        this.transport = new Transport(options);
+        const api = bind((operation, input, callOptions) => this.call(operation, input, callOptions));
+        this.files = { ...api.files, each: (query = {}, callOptions) => pages(api.files.list, query, callOptions) };
+        this.folders = api.folders;
+        this.tree = { ...api.tree, walk: (query = {}, callOptions) => new Walk(api.tree.list, query, callOptions) };
+        this.changes = { ...api.changes, follow: (cursor, followOptions = {}) => follow(api.changes.list, cursor ?? null, followOptions) };
+        this.uploads = api.uploads;
+        this.usage = api.usage;
+        this.tokens = api.tokens;
+    }
+    /** Any endpoint, by its operation name. The functions above are this, typed. */
+    async call(operation, input = {}, options = {}) {
+        const op = OPERATIONS[operation];
+        const path = op.path.replace(/:(\w+)/g, (_, name) => {
+            const value = input.params?.[name];
+            if (!value)
+                throw new TypeError(`${operation} needs ${name}.`);
+            return encodeURIComponent(value);
+        });
+        const res = await this.transport.send({
+            method: op.method,
+            path,
+            query: input.query,
+            body: input.body,
+            idempotent: op.idempotent,
+            repeatable: op.repeatable,
+            options,
+        });
+        return res.body;
+    }
+    /**
+     * Send a file straight to storage in parts, and make it a file. The API is
+     * told what is coming and when it has all arrived; the bytes go from here
+     * to storage. See UploadOptions, and UploadStopped for carrying on.
+     */
+    upload(source, options = {}) {
+        return upload({ ...this.uploads, file: this.files.get }, this.transport.fetch, source, options);
+    }
+    /**
+     * Fetch a file straight from storage into `sink`: as up to six ranges at
+     * once for a large one. One download address is asked for, so the file is
+     * counted toward the month's downloads once. A file under Shard Protection
+     * has no address, and is read through its map instead (files.shards), each
+     * fragment checked as it arrives. Resolves to the file.
+     */
+    async download(id, sink, options = {}) {
+        try {
+            const { item } = await this.files.get(id, { signal: options.signal });
+            await this.fetchInto(item, sink, options);
+            await sink.close?.();
+            return item;
+        }
+        catch (e) {
+            await sink.abort?.(e).catch(() => undefined);
+            throw e;
+        }
+    }
+    /** A file's bytes, in memory: for a file small enough to hold. */
+    async bytes(id, options = {}) {
+        const { item } = await this.files.get(id, { signal: options.signal });
+        const bytes = new Uint8Array(item.sizeBytes);
+        await this.fetchInto(item, { write: async (position, data) => bytes.set(data, position) }, options);
+        return bytes;
+    }
+    async fetchInto(item, sink, options) {
+        if (item.kind !== "file")
+            throw new FileFreakError(400, "not_a_file", "A folder cannot be downloaded as one file.");
+        // Nothing to fetch, and an address would count a download of nothing.
+        if (item.sizeBytes === 0)
+            return;
+        const { signal, onProgress } = options;
+        const progress = onProgress && ((written) => onProgress(written, item.sizeBytes));
+        await asItIsKept(item.protected, async () => {
+            const { url } = await this.files.downloadUrl(item.id, {}, { signal });
+            await fetchRanges({ url, sizeBytes: item.sizeBytes, sink, signal, progress, ranges: options.ranges, fetch: this.transport.fetch, sleep: (ms) => sleep(ms, signal) });
+        }, () => readProtected({
+            pages: (from, pageSignal) => this.files.shards(item.id, { from }, { signal: pageSignal }),
+            sink,
+            sizeBytes: item.sizeBytes,
+            signal,
+            progress,
+            fetch: this.transport.fetch,
+        }));
+    }
+}
+/** Every item in a listing, a page at a time. */
+async function* pages(list, query, options) {
+    let cursor = null;
+    do {
+        const page = await list(cursor ? { ...query, cursor } : query, options);
+        yield* page.items;
+        cursor = page.nextCursor;
+    } while (cursor);
+}
+async function* follow(list, from, options) {
+    const { wait = FOLLOW_WAIT_SECONDS, limit, ...callOptions } = options;
+    let cursor = from;
+    for (;;) {
+        const query = { wait };
+        if (cursor)
+            query.cursor = cursor;
+        if (limit)
+            query.limit = limit;
+        const asked = Date.now();
+        const answer = await list(query, callOptions);
+        if (answer.changes.length > 0)
+            yield answer;
+        cursor = answer.cursor;
+        // Nothing, at once: asked with no wait, or the API cut the wait short
+        // (it does when it is stopping). Asking again at once would go round as
+        // fast as the line allows.
+        const early = MIN_ASK_MS - (Date.now() - asked);
+        if (answer.changes.length === 0 && !answer.more && early > 0)
+            await sleep(early, callOptions.signal);
+    }
+}