@elaraai/e3-cloud-files 0.0.0-stage → 1.0.92

package/LICENSE.md

@@ -0,0 +1,54 @@
+# Business Source License 1.1
+
+Copyright (c) 2025 Elara AI Pty Ltd
+
+## License
+
+**Licensor:** Elara AI Pty Ltd
+
+**Licensed Work:** @elaraai/e3-cloud-files
+
+**Change Date:** Four years from the date of each release
+
+**Change License:** AGPL-3.0
+
+## Terms
+
+The Licensed Work is provided under the terms of the Business Source License 1.1 as detailed below.
+
+### Grant of Rights
+
+The Licensor grants you the right to copy, modify, create derivative works, redistribute, and make non-production use of the Licensed Work.
+
+### Production Use Limitation
+
+**"Production Use"** means any use by or on behalf of a for-profit entity, other than for evaluation, testing, or development purposes.
+
+Production Use requires a separate commercial license from the Licensor.
+
+### Change Date
+
+On the Change Date (four years after each release), the Licensed Work will be made available under the Change License (AGPL-3.0).
+
+### No Warranty
+
+THE LICENSED WORK IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND. THE LICENSOR DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE.
+
+### Package-Specific Licenses
+
+Individual packages within this repository may have different licensing terms. See the LICENSE.md file in each package directory for details.
+
+## Commercial Licensing
+
+To obtain a commercial license for Production Use, contact:
+
+**Email:** support@elara.ai
+**Website:** https://elaraai.com
+
+## Governing Law
+
+This license is governed by the laws of New South Wales, Australia.
+
+---
+
+*Elara AI Pty Ltd*

package/README.md

@@ -1,3 +1,287 @@
-# Temporary Holding Version
+# @elaraai/e3-cloud-files
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+Files for e3 cloud: a solution's inputs that come from files in a bucket
+(`s3://bucket/prefix/`).
+The solution says in its code which inputs a bucket's files fill. Each
+workspace says where the files are. `e3-cloud files sync` brings in what
+changed, the same way on a local repository and on a deployed workspace.
+
+- **Your data stays in the bucket.** Locally, a sync reads the bucket with
+  your own cloud credentials. The bytes land only in the local e3 repository,
+  never in your project.
+- **A file arrives as it is**, a `Blob`. A task decodes it in East: as East's
+  JSON, as CSV, or as text.
+- **Nothing polls.** A sync runs when someone asks for it, and reads only the
+  files that changed since the last one.
+
+Its version is the @elaraai lockstep it is built against: install the one of
+the same version as your east and e3.
+
+## Install
+
+```bash
+npm install @elaraai/e3-cloud-files@<your e3's version>
+```
+
+Its peers are the same lockstep's `@elaraai/east` and `@elaraai/e3`. The
+`e3-cloud` CLI (`@elaraai/e3-cloud-cli`) deploys and syncs.
+
+## Declaring the files
+
+A **source** is a place files come from, named in your code. Each workspace
+binds it to a bucket and prefix. Its shapes declare the inputs its files
+fill:
+
+```ts
+// src/files.ts
+import { Files } from '@elaraai/e3-cloud-files';
+
+/** Where the partners' files are: each workspace binds it to a bucket and prefix. */
+export const upload = Files.source('upload');
+
+/** One file: today's exchange rates. */
+export const rates = upload.file('rates', { key: 'rates.json' });
+/** The newest stock snapshot: named by date, so the greatest key is the latest. */
+export const stock = upload.latest('stock', { prefix: 'stock/', include: '*.json' });
+/** Every order file, keyed by its name. */
+export const orders = upload.folder('orders', { prefix: 'orders/', include: '*.json' });
+/** The order files' sizes, times and ETags, without their bytes. */
+export const orderFiles = upload.listing('order_files', { prefix: 'orders/', include: '*.json' });
+
+export default Files.manifest([upload]);
+```
+
+| Shape | Selects | The input's type | Largest file |
+|---|---|---|---|
+| `file(input, { key })` | One key | `Blob` | 1 GiB less 64 KiB |
+| `latest(input, { prefix, include? })` | The greatest key under the prefix that matches | `Blob` | 1 GiB less 64 KiB |
+| `folder(input, { prefix, include? })` | Every key under the prefix that matches, by its path under the prefix | `Dict<String, Blob>` | 64 MiB less 64 KiB each |
+| `listing(input, { prefix, include? })` | The same keys' size, time and ETag, without their bytes | `Dict<String, Struct{size, modified, etag}>` | none |
+
+- **Each shape returns an ordinary `e3.input`.** It has no default, so it
+  stays unassigned until a sync sets it. A task takes it like any input, and
+  a task that needs it waits until the first sync, as with any unassigned
+  input.
+- **Keys and prefixes are relative to the binding.** The code never names a
+  bucket. A prefix is `''` (everything the binding holds) or ends with `/`.
+- **`include` is a glob** over each file's path under the prefix: `*` matches
+  within one folder, `**` across folders, `?` one character, `[abc]` or
+  `[a-z]` one of a set (`[!abc]` any but those), and `{csv,json}` any of its
+  alternatives, each a glob itself (`*.{csv,json}`). `*.json` takes only the
+  prefix's own files, and lists only its own folder; `**/*.json` takes its
+  subfolders' too, their paths naming the subfolder (`2026/10/a.json`).
+- **Mistakes are refused as they are declared** (`FilesDeclarationError`, one
+  line per problem): a name e3 refuses, a key that isn't relative, a glob
+  that can't match, and in `Files.manifest` two sources of one name or two
+  shapes filling one input.
+
+## Decoding in a task
+
+A task decodes what arrives. The package is a package, as always:
+
+```ts
+// src/package.ts
+import { East, ArrayType, BlobType, FloatType, StringType, StructType } from '@elaraai/east';
+import e3 from '@elaraai/e3';
+import { FilesFolderType } from '@elaraai/e3-cloud-files';
+import { rates, orders } from './files.js';
+
+const Rate = StructType({ currency: StringType, rate: FloatType });
+const Order = StructType({ id: StringType, sku: StringType, currency: StringType, amount: FloatType });
+
+const decodeRates = East.function([BlobType], ArrayType(Rate), (_$, file) =>
+  file.decodeUtf8().parseJson(ArrayType(Rate)));
+
+const decodeOrders = East.function([FilesFolderType], ArrayType(Order), (_$, files) =>
+  files.flattenToArray((_$, file) => file.decodeUtf8().parseJson(ArrayType(Order))));
+
+export const rateTable = e3.task('rate_table', [rates], decodeRates, { runner });
+export const allOrders = e3.task('all_orders', [orders], decodeOrders, { runner });
+
+export default e3.package('orders', '1.0.0', rateTable, allOrders);
+```
+
+- **CSV:** `file.decodeCsv(Row, { delimiter: ',' })`, the header naming the
+  struct's fields.
+- **Text:** `file.decodeUtf8().split('\n')`.
+- **A folder** is in its paths' order. `flattenToArray` joins its files'
+  rows; its key is each file's path, if a row should say where it came from.
+- **A listing** lets a task choose files by name, size or time without
+  reading them.
+
+The plugin's example index holds one example of each (search
+`e3-cloud-files`).
+
+## Deploying and syncing
+
+The package deploys as always. The declarations deploy beside it, are checked
+against it, and each source is bound:
+
+```bash
+e3 workspace deploy ./repo dev --from-source src/package.ts
+e3-cloud files deploy ./repo dev src/files.ts
+e3-cloud files bind   ./repo dev upload s3://partner-drop/orders-team/ --profile partner --latest 30
+e3-cloud files sync   ./repo dev
+e3 dataflow run ./repo dev
+```
+
+- **`<repo>` is a local repository's path or a repository URL**
+  (`https://e3.example.com/repos/acme`). The commands are the same for both.
+- **`files deploy`** runs the module (its default export is
+  `Files.manifest(...)`) and checks the declarations against the package the
+  workspace runs. Each mapping's input must be in the package, writable, and
+  exactly its shape's type. `--dry-run` checks and keeps nothing.
+- **`files bind`** says where a source's files are for this workspace:
+  `s3://bucket/prefix/`. `--region` gives the bucket's region when it isn't
+  your configured one. On a local repository only: `--profile` names the
+  profile to read with (one of `~/.aws/config`'s), and `--latest n` takes
+  only each folder's n greatest keys, which caps what comes down to your
+  machine.
+- **`files sync`** lists each source and compares every file with what its
+  input took last, by ETag and size. It reads only what is new or changed,
+  writes only the inputs whose files changed, and says what it did per input.
+  `--input` syncs only those inputs; `--dry-run` lists what a sync would take
+  and takes nothing. A sync doesn't run the dataflow: `e3 dataflow run` does, and the
+  sync prints the command when anything changed.
+- **`files status`** shows the declarations, where each source is bound, and
+  the last sync.
+
+A sync is safe to repeat and to stop. Each input is remembered only once it
+is written, so a sync that stops part way takes the rest next time.
+
+### What a sync takes again whole
+
+A sync trusts what an input took last only while the input still holds the
+value the sync wrote, and its declaration and binding are the ones it was
+taken under, which it remembers with it. It takes an input's files again
+whole when:
+
+- a deploy reset the input, or someone set it by hand. A package deploy resets
+  inputs unless told to keep them: `e3 workspace deploy … --inputs
+  keep-edited` keeps synced inputs while their type is unchanged;
+- the input's declaration changed (`files deploy` says which), even when a
+  sync under the old one was still writing it;
+- its source was bound somewhere else (`files bind` says which).
+
+### Files a sync can't read
+
+A file that can't be read is refused, named in the report, and the sync goes
+on with the rest: a file archived to a storage class that must be restored
+first, one encrypted with a key the reader may not use, one that changed or
+went between the listing and its read. The input keeps what it held of it,
+and the next sync tries it again. Any other failure (the bucket unreachable,
+throttled) stops the sync, which is safe to run again.
+
+### Staging and production
+
+On a deployment, a workspace may have a staging: another workspace where
+changes are tried before they go live (`e3-cloud staging`). The declarations
+are part of the solution's code, so they travel as its package and apps do.
+Where the files are is each workspace's own.
+
+- **A release** (`e3-cloud release <url> <staging> <production>`) gives
+  production the staging's declarations, with its package and apps. Its plan
+  says what happens to each input: added, changed, unchanged or removed. The
+  release is refused while the declarations don't fit the package production
+  will run, or read a source production hasn't bound. Production's next sync
+  takes an input whose declaration changed again whole. A release syncs
+  nothing: sync production after it.
+- **Production's bindings are its own**: a release never changes them. Bind a
+  new source in production before the release that brings the declarations
+  reading it; `files bind` takes a source no declaration reads yet.
+- **A protected workspace refuses `files deploy`**: its declarations change
+  only by a release from its staging, or by a rollback. In an emergency an
+  admin can force a deploy past it (`--force`). Its history records the
+  force, and the next release from staging undoes it.
+- **A rollback** (`e3-cloud rollback <url> <production>`) restores the
+  declarations of the entry it goes back to, with that entry's package and
+  apps.
+- **A copy** of production, such as a staging started from it, gives the
+  target everything production has of its Files. That is its declarations,
+  its bindings, its last sync and what each input took. So a staging
+  rehearses on production's files, and its first sync takes only what
+  changed since production's.
+- The workspace's history (`e3-cloud history`) lists every deploy of
+  declarations, every binding moved, and every forced change.
+
+```bash
+# Try a change on staging
+e3-cloud files deploy https://e3.example.com/repos/acme staging src/files.ts
+e3-cloud files sync   https://e3.example.com/repos/acme staging
+e3 dataflow run https://e3.example.com/repos/acme staging
+
+# Then production: bind any new source first, release, then sync
+e3-cloud files bind https://e3.example.com/repos/acme production invoices s3://drop/acme/invoices/
+e3-cloud release    https://e3.example.com/repos/acme staging production
+e3-cloud files sync https://e3.example.com/repos/acme production
+e3 dataflow run https://e3.example.com/repos/acme production
+```
+
+### Large folders
+
+A folder is stored as e3 stores any large collection, in pieces each holding
+the files between two keys. A sync that changes a few files rewrites only the
+pieces holding them, and carries every other piece over unread.
+
+A folder's new and changed files are taken in batches of at most 256 MiB:
+each batch is read, written into the folder and remembered before the next
+is read. So a sync holds one batch of files and the pieces it changes,
+however large the folder; the first sync of 6 GB of files holds 256 MiB at a
+time, and a sync stopped part way takes the rest next time.
+
+A `file` or `latest` input is held whole while it is written: a 1 GiB file
+needs some 3 GiB of memory.
+
+### On a local repository
+
+The repository keeps each workspace's Files in
+`<repo>/.e3-cloud/files/<workspace>.json`: the declarations, the bindings,
+what each input took last, and the last sync's report. It belongs to the
+repository, never to the project. Removing the repository removes it, along
+with every byte the syncs took in.
+
+A sync reads with the binding's `--profile`, else the SDK's usual chain
+(`AWS_PROFILE`, the environment, SSO). An expired SSO login says which source
+and profile it was: `aws sso login --profile <name>` renews it.
+
+## The checks
+
+`@elaraai/e3-cloud-files/check` holds the checks without the authoring:
+
+- `checkFiles(manifest, inputs)`: the declarations against a package's inputs
+  (`readPackageInputs(read, packageHash)` reads them from its objects);
+- `checkBindings(manifest, bindings, policy)`: bindings against the
+  declarations, and in the cloud against the sources the deployment grants.
+
+Both answer `FilesProblem` values (from `@elaraai/e3-cloud-types`), each
+rendered as one line by `filesProblemText`.
+
+## The East types
+
+In `@elaraai/e3-cloud-types`, re-exported here:
+
+- `FilesManifestType`: the sources' names, and each mapping's input, source
+  and shape (`FilesShapeType`: `file`, `latest`, `folder`, `listing`).
+- `FilesBindingType`: a source's URL, region, and, locally, profile and
+  `latest`.
+- `FilesSyncReportType`: per input, the files new, changed, removed,
+  unchanged and refused, the bytes taken in, and whether it was written.
+- `FilesStatusType`, `FilesProblemType`.
+- The inputs' types: `FilesFileType` (`Blob`), `FilesFolderType`
+  (`Dict<String, Blob>`), `FilesListingType`.
+
+## Entries
+
+| Entry | Holds |
+|---|---|
+| `@elaraai/e3-cloud-files` | Authoring: `Files.source`, the shapes, `Files.manifest`, and the types |
+| `@elaraai/e3-cloud-files/check` | The checks, without the authoring |
+| `@elaraai/e3-cloud-files/sync` | The sync (`syncFiles`) over what reads, writes and remembers, and those three in memory for tests |
+| `@elaraai/e3-cloud-files/storage` | The sync's writer over an e3 store (needs `@elaraai/e3-core`) |
+| `@elaraai/e3-cloud-files/s3` | The sync's reader of a bucket (needs `@aws-sdk/client-s3`) |
+| `@elaraai/e3-cloud-files/examples/*` | The worked example (`orders`) and the decoding examples (`decode`) |
+
+## License
+
+[BSL 1.1](./LICENSE.md). Production use needs a commercial license from Elara
+AI Pty Ltd. Each release becomes AGPL-3.0 four years after it is released.

package/dist/src/authoring.d.ts

@@ -0,0 +1,114 @@
+/**
+ * Copyright (c) 2025 Elara AI Pty Ltd
+ * Licensed under BSL 1.1. See LICENSE for details.
+ */
+/**
+ * Declaring which inputs come from files.
+ *
+ * ```ts
+ * export const upload = Files.source('upload');
+ * export const rates  = upload.file('rates', { key: 'rates.json' });
+ * export const stock  = upload.latest('stock', { prefix: 'stock/' });
+ * export const orders = upload.folder('orders', { prefix: 'orders/', include: '*.json' });
+ * export default Files.manifest([upload]);
+ * ```
+ *
+ * Each declaration is an ordinary `e3.input` with no default, so the package
+ * is plain e3 and tasks take these inputs as any other. A sync fills them;
+ * until the first, they are unassigned. What each source's files fill is the
+ * {@link Files.manifest}, which `e3-cloud files deploy` deploys beside the
+ * package.
+ *
+ * @packageDocumentation
+ */
+import e3 from '@elaraai/e3';
+import { type EastType } from '@elaraai/east';
+import { FilesFileType, FilesFolderType, FilesListingType, type FilesManifest, type FilesMapping, type FilesProblem } from '@elaraai/e3-cloud-types';
+/** The input a declaration makes: an `e3.input` of its shape's type, with no default. */
+export type FilesInput<Name extends string, T extends EastType> = ReturnType<typeof e3.input<Name, T>>;
+/** The selection of `latest`, `folder` and `listing`: a prefix, and a glob over the paths under it. */
+export interface FilesSelection {
+    /** The folder, relative to where the workspace binds the source: `''` for all of it, else ending with `/` */
+    readonly prefix: string;
+    /** A glob over paths relative to the prefix: `*` within a segment, `**` across segments, `?` one character, `[a-z]` one of a set, `{csv,json}` any of its alternatives */
+    readonly include?: string;
+}
+/** Thrown when declarations are not well-formed, naming every problem. */
+export declare class FilesDeclarationError extends Error {
+    /** Every problem */
+    readonly problems: readonly FilesProblem[];
+    constructor(problems: readonly FilesProblem[]);
+}
+/**
+ * A source of files: a name each workspace binds to a bucket and a prefix,
+ * and the inputs its files fill.
+ */
+export declare class FilesSource {
+    #private;
+    /** The source's name */
+    readonly name: string;
+    /** @internal Use {@link Files.source}. */
+    constructor(name: string);
+    /** The inputs this source fills, in the order they were declared. */
+    get mappings(): readonly FilesMapping[];
+    /**
+     * One file, as a `Blob`: the input `.inputs.<input>`.
+     *
+     * @param input - The input's name
+     * @param spec - `key`: the file, relative to where the workspace binds the source
+     * @returns The input, for tasks to take
+     */
+    file<Name extends string>(input: Name, spec: {
+        readonly key: string;
+    }): FilesInput<Name, typeof FilesFileType>;
+    /**
+     * The newest file under a prefix, as a `Blob`: the greatest key that matches
+     * `include`, so files named by their time pick the latest.
+     *
+     * @param input - The input's name
+     * @param spec - The prefix, and a glob over the paths under it
+     * @returns The input, for tasks to take
+     */
+    latest<Name extends string>(input: Name, spec: FilesSelection): FilesInput<Name, typeof FilesFileType>;
+    /**
+     * Every file under a prefix that matches `include`, as a
+     * `Dict<String, Blob>` keyed by each file's path relative to the prefix.
+     *
+     * @param input - The input's name
+     * @param spec - The prefix, and a glob over the paths under it
+     * @returns The input, for tasks to take
+     */
+    folder<Name extends string>(input: Name, spec: FilesSelection): FilesInput<Name, typeof FilesFolderType>;
+    /**
+     * The files under a prefix that match `include`, without their bytes: each
+     * one's size, time and ETag, keyed by its path relative to the prefix, so a
+     * task can pick files by name or time.
+     *
+     * @param input - The input's name
+     * @param spec - The prefix, and a glob over the paths under it
+     * @returns The input, for tasks to take
+     */
+    listing<Name extends string>(input: Name, spec: FilesSelection): FilesInput<Name, typeof FilesListingType>;
+}
+/** Declaring files. */
+export declare const Files: {
+    /**
+     * A source of files, which each workspace binds to a bucket and a prefix.
+     *
+     * @param name - Its name, unique among a solution's sources
+     * @returns The source, whose methods declare the inputs it fills
+     * @throws {FilesDeclarationError} When the name is one e3 refuses
+     */
+    source(name: string): FilesSource;
+    /**
+     * The declarations `e3-cloud files deploy` deploys: the sources, and the
+     * inputs their files fill.
+     *
+     * @param sources - Every source
+     * @returns The manifest, an East value of `FilesManifestType`
+     * @throws {FilesDeclarationError} When two sources share a name, or two
+     *   mappings fill one input
+     */
+    manifest(sources: readonly FilesSource[]): FilesManifest;
+};
+//# sourceMappingURL=authoring.d.ts.map

package/dist/src/authoring.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"authoring.d.ts","sourceRoot":"","sources":["../../src/authoring.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;GAkBG;AAEH,OAAO,EAAE,MAAM,aAAa,CAAC;AAC7B,OAAO,EAAuB,KAAK,QAAQ,EAAE,MAAM,eAAe,CAAC;AAEnE,OAAO,EACL,aAAa,EAAE,eAAe,EAAE,gBAAgB,EAEhD,KAAK,aAAa,EAAE,KAAK,YAAY,EAAE,KAAK,YAAY,EACzD,MAAM,yBAAyB,CAAC;AAGjC,yFAAyF;AACzF,MAAM,MAAM,UAAU,CAAC,IAAI,SAAS,MAAM,EAAE,CAAC,SAAS,QAAQ,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;AAEvG,uGAAuG;AACvG,MAAM,WAAW,cAAc;IAC7B,6GAA6G;IAC7G,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,0KAA0K;IAC1K,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED,0EAA0E;AAC1E,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,oBAAoB;IACpB,QAAQ,CAAC,QAAQ,EAAE,SAAS,YAAY,EAAE,CAAC;gBAE/B,QAAQ,EAAE,SAAS,YAAY,EAAE;CAK9C;AA8BD;;;GAGG;AACH,qBAAa,WAAW;;IACtB,wBAAwB;IACxB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAGtB,0CAA0C;gBAC9B,IAAI,EAAE,MAAM;IAKxB,qEAAqE;IACrE,IAAI,QAAQ,IAAI,SAAS,YAAY,EAAE,CAEtC;IASD;;;;;;OAMG;IACH,IAAI,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE;QAAE,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAA;KAAE,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAM9G;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAItG;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,eAAe,CAAC;IAIxG;;;;;;;;OAQG;IACH,OAAO,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,gBAAgB,CAAC;CAG3G;AAED,uBAAuB;AACvB,eAAO,MAAM,KAAK;IAChB;;;;;;OAMG;iBACU,MAAM,GAAG,WAAW;IAIjC;;;;;;;;OAQG;sBACe,SAAS,WAAW,EAAE,GAAG,aAAa;CAmBzD,CAAC"}

package/dist/src/authoring.js

@@ -0,0 +1,178 @@
+/**
+ * Copyright (c) 2025 Elara AI Pty Ltd
+ * Licensed under BSL 1.1. See LICENSE for details.
+ */
+/**
+ * Declaring which inputs come from files.
+ *
+ * ```ts
+ * export const upload = Files.source('upload');
+ * export const rates  = upload.file('rates', { key: 'rates.json' });
+ * export const stock  = upload.latest('stock', { prefix: 'stock/' });
+ * export const orders = upload.folder('orders', { prefix: 'orders/', include: '*.json' });
+ * export default Files.manifest([upload]);
+ * ```
+ *
+ * Each declaration is an ordinary `e3.input` with no default, so the package
+ * is plain e3 and tasks take these inputs as any other. A sync fills them;
+ * until the first, they are unassigned. What each source's files fill is the
+ * {@link Files.manifest}, which `e3-cloud files deploy` deploys beside the
+ * package.
+ *
+ * @packageDocumentation
+ */
+import e3 from '@elaraai/e3';
+import { none, some, variant } from '@elaraai/east';
+import { nameProblem } from '@elaraai/e3-types';
+import { FilesFileType, FilesFolderType, FilesListingType, filesProblemText, } from '@elaraai/e3-cloud-types';
+import { includeProblem, keyProblem, prefixProblem } from './paths.js';
+/** Thrown when declarations are not well-formed, naming every problem. */
+export class FilesDeclarationError extends Error {
+    /** Every problem */
+    problems;
+    constructor(problems) {
+        super(`The files declarations are not well-formed:\n${problems.map((p) => `  - ${filesProblemText(p)}`).join('\n')}`);
+        this.name = 'FilesDeclarationError';
+        this.problems = problems;
+    }
+}
+/** Throws a declaration's problems, if it has any. */
+function refuse(problems) {
+    if (problems.length > 0)
+        throw new FilesDeclarationError(problems);
+}
+/** A name's problem, as e3 names a dataset's path segment. */
+function nameProblems(kind, name) {
+    const reason = nameProblem('workspace', name);
+    return reason === null ? [] : [variant('invalid_name', { kind, name, reason })];
+}
+/** A selection's problems. */
+function selectionProblems(input, selection) {
+    const problems = [];
+    const prefix = prefixProblem(selection.prefix);
+    if (prefix !== null)
+        problems.push(variant('invalid_key', { input, key: selection.prefix, reason: prefix }));
+    if (selection.include !== undefined) {
+        const include = includeProblem(selection.include);
+        if (include !== null)
+            problems.push(variant('invalid_include', { input, include: selection.include, reason: include }));
+    }
+    return problems;
+}
+/** A selection's shape fields. */
+function selected(selection) {
+    return { prefix: selection.prefix, include: selection.include === undefined ? none : some(selection.include) };
+}
+/**
+ * A source of files: a name each workspace binds to a bucket and a prefix,
+ * and the inputs its files fill.
+ */
+export class FilesSource {
+    /** The source's name */
+    name;
+    #mappings = [];
+    /** @internal Use {@link Files.source}. */
+    constructor(name) {
+        refuse(nameProblems('source', name));
+        this.name = name;
+    }
+    /** The inputs this source fills, in the order they were declared. */
+    get mappings() {
+        return this.#mappings;
+    }
+    /** Records a mapping and makes its input. */
+    #declare(input, type, shape, problems) {
+        refuse([...nameProblems('input', input), ...problems]);
+        this.#mappings.push({ input, source: this.name, shape });
+        return e3.input(input, type);
+    }
+    /**
+     * One file, as a `Blob`: the input `.inputs.<input>`.
+     *
+     * @param input - The input's name
+     * @param spec - `key`: the file, relative to where the workspace binds the source
+     * @returns The input, for tasks to take
+     */
+    file(input, spec) {
+        const reason = keyProblem(spec.key);
+        const problems = reason === null ? [] : [variant('invalid_key', { input, key: spec.key, reason })];
+        return this.#declare(input, FilesFileType, variant('file', { key: spec.key }), problems);
+    }
+    /**
+     * The newest file under a prefix, as a `Blob`: the greatest key that matches
+     * `include`, so files named by their time pick the latest.
+     *
+     * @param input - The input's name
+     * @param spec - The prefix, and a glob over the paths under it
+     * @returns The input, for tasks to take
+     */
+    latest(input, spec) {
+        return this.#declare(input, FilesFileType, variant('latest', selected(spec)), selectionProblems(input, spec));
+    }
+    /**
+     * Every file under a prefix that matches `include`, as a
+     * `Dict<String, Blob>` keyed by each file's path relative to the prefix.
+     *
+     * @param input - The input's name
+     * @param spec - The prefix, and a glob over the paths under it
+     * @returns The input, for tasks to take
+     */
+    folder(input, spec) {
+        return this.#declare(input, FilesFolderType, variant('folder', selected(spec)), selectionProblems(input, spec));
+    }
+    /**
+     * The files under a prefix that match `include`, without their bytes: each
+     * one's size, time and ETag, keyed by its path relative to the prefix, so a
+     * task can pick files by name or time.
+     *
+     * @param input - The input's name
+     * @param spec - The prefix, and a glob over the paths under it
+     * @returns The input, for tasks to take
+     */
+    listing(input, spec) {
+        return this.#declare(input, FilesListingType, variant('listing', selected(spec)), selectionProblems(input, spec));
+    }
+}
+/** Declaring files. */
+export const Files = {
+    /**
+     * A source of files, which each workspace binds to a bucket and a prefix.
+     *
+     * @param name - Its name, unique among a solution's sources
+     * @returns The source, whose methods declare the inputs it fills
+     * @throws {FilesDeclarationError} When the name is one e3 refuses
+     */
+    source(name) {
+        return new FilesSource(name);
+    },
+    /**
+     * The declarations `e3-cloud files deploy` deploys: the sources, and the
+     * inputs their files fill.
+     *
+     * @param sources - Every source
+     * @returns The manifest, an East value of `FilesManifestType`
+     * @throws {FilesDeclarationError} When two sources share a name, or two
+     *   mappings fill one input
+     */
+    manifest(sources) {
+        const problems = [];
+        const names = new Set();
+        const inputs = new Set();
+        const mappings = [];
+        for (const source of sources) {
+            if (names.has(source.name)) {
+                problems.push(variant('invalid_name', { kind: 'source', name: source.name, reason: 'is declared twice' }));
+            }
+            names.add(source.name);
+            for (const mapping of source.mappings) {
+                if (inputs.has(mapping.input))
+                    problems.push(variant('duplicate_input', { input: mapping.input }));
+                inputs.add(mapping.input);
+                mappings.push(mapping);
+            }
+        }
+        refuse(problems);
+        return { sources: [...names], mappings };
+    },
+};
+//# sourceMappingURL=authoring.js.map

package/dist/src/authoring.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"authoring.js","sourceRoot":"","sources":["../../src/authoring.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;GAkBG;AAEH,OAAO,EAAE,MAAM,aAAa,CAAC;AAC7B,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,OAAO,EAAiB,MAAM,eAAe,CAAC;AACnE,OAAO,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAChD,OAAO,EACL,aAAa,EAAE,eAAe,EAAE,gBAAgB,EAChD,gBAAgB,GAEjB,MAAM,yBAAyB,CAAC;AACjC,OAAO,EAAE,cAAc,EAAE,UAAU,EAAE,aAAa,EAAE,MAAM,YAAY,CAAC;AAavE,0EAA0E;AAC1E,MAAM,OAAO,qBAAsB,SAAQ,KAAK;IAC9C,oBAAoB;IACX,QAAQ,CAA0B;IAE3C,YAAY,QAAiC;QAC3C,KAAK,CAAC,gDAAgD,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,OAAO,gBAAgB,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;QACtH,IAAI,CAAC,IAAI,GAAG,uBAAuB,CAAC;QACpC,IAAI,CAAC,QAAQ,GAAG,QAAQ,CAAC;IAC3B,CAAC;CACF;AAED,sDAAsD;AACtD,SAAS,MAAM,CAAC,QAAiC;IAC/C,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC;QAAE,MAAM,IAAI,qBAAqB,CAAC,QAAQ,CAAC,CAAC;AACrE,CAAC;AAED,8DAA8D;AAC9D,SAAS,YAAY,CAAC,IAAwB,EAAE,IAAY;IAC1D,MAAM,MAAM,GAAG,WAAW,CAAC,WAAW,EAAE,IAAI,CAAC,CAAC;IAC9C,OAAO,MAAM,KAAK,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,cAAc,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC;AAClF,CAAC;AAED,8BAA8B;AAC9B,SAAS,iBAAiB,CAAC,KAAa,EAAE,SAAyB;IACjE,MAAM,QAAQ,GAAmB,EAAE,CAAC;IACpC,MAAM,MAAM,GAAG,aAAa,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;IAC/C,IAAI,MAAM,KAAK,IAAI;QAAE,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,EAAE,EAAE,KAAK,EAAE,GAAG,EAAE,SAAS,CAAC,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC;IAC7G,IAAI,SAAS,CAAC,OAAO,KAAK,SAAS,EAAE,CAAC;QACpC,MAAM,OAAO,GAAG,cAAc,CAAC,SAAS,CAAC,OAAO,CAAC,CAAC;QAClD,IAAI,OAAO,KAAK,IAAI;YAAE,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,iBAAiB,EAAE,EAAE,KAAK,EAAE,OAAO,EAAE,SAAS,CAAC,OAAO,EAAE,MAAM,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC;IAC1H,CAAC;IACD,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,kCAAkC;AAClC,SAAS,QAAQ,CAAC,SAAyB;IACzC,OAAO,EAAE,MAAM,EAAE,SAAS,CAAC,MAAM,EAAE,OAAO,EAAE,SAAS,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,EAAE,CAAC;AACjH,CAAC;AAED;;;GAGG;AACH,MAAM,OAAO,WAAW;IACtB,wBAAwB;IACf,IAAI,CAAS;IACb,SAAS,GAAmB,EAAE,CAAC;IAExC,0CAA0C;IAC1C,YAAY,IAAY;QACtB,MAAM,CAAC,YAAY,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC,CAAC;QACrC,IAAI,CAAC,IAAI,GAAG,IAAI,CAAC;IACnB,CAAC;IAED,qEAAqE;IACrE,IAAI,QAAQ;QACV,OAAO,IAAI,CAAC,SAAS,CAAC;IACxB,CAAC;IAED,6CAA6C;IAC7C,QAAQ,CAA0C,KAAW,EAAE,IAAO,EAAE,KAAiB,EAAE,QAAwB;QACjH,MAAM,CAAC,CAAC,GAAG,YAAY,CAAC,OAAO,EAAE,KAAK,CAAC,EAAE,GAAG,QAAQ,CAAC,CAAC,CAAC;QACvD,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,EAAE,KAAK,EAAE,MAAM,EAAE,IAAI,CAAC,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC;QACzD,OAAO,EAAE,CAAC,KAAK,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;IAC/B,CAAC;IAED;;;;;;OAMG;IACH,IAAI,CAAsB,KAAW,EAAE,IAA8B;QACnE,MAAM,MAAM,GAAG,UAAU,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QACpC,MAAM,QAAQ,GAAmB,MAAM,KAAK,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,aAAa,EAAE,EAAE,KAAK,EAAE,GAAG,EAAE,IAAI,CAAC,GAAG,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC;QACnH,OAAO,IAAI,CAAC,QAAQ,CAAC,KAAK,EAAE,aAAa,EAAE,OAAO,CAAC,MAAM,EAAE,EAAE,GAAG,EAAE,IAAI,CAAC,GAAG,EAAE,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC3F,CAAC;IAED;;;;;;;OAOG;IACH,MAAM,CAAsB,KAAW,EAAE,IAAoB;QAC3D,OAAO,IAAI,CAAC,QAAQ,CAAC,KAAK,EAAE,aAAa,EAAE,OAAO,CAAC,QAAQ,EAAE,QAAQ,CAAC,IAAI,CAAC,CAAC,EAAE,iBAAiB,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC,CAAC;IAChH,CAAC;IAED;;;;;;;OAOG;IACH,MAAM,CAAsB,KAAW,EAAE,IAAoB;QAC3D,OAAO,IAAI,CAAC,QAAQ,CAAC,KAAK,EAAE,eAAe,EAAE,OAAO,CAAC,QAAQ,EAAE,QAAQ,CAAC,IAAI,CAAC,CAAC,EAAE,iBAAiB,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC,CAAC;IAClH,CAAC;IAED;;;;;;;;OAQG;IACH,OAAO,CAAsB,KAAW,EAAE,IAAoB;QAC5D,OAAO,IAAI,CAAC,QAAQ,CAAC,KAAK,EAAE,gBAAgB,EAAE,OAAO,CAAC,SAAS,EAAE,QAAQ,CAAC,IAAI,CAAC,CAAC,EAAE,iBAAiB,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC,CAAC;IACpH,CAAC;CACF;AAED,uBAAuB;AACvB,MAAM,CAAC,MAAM,KAAK,GAAG;IACnB;;;;;;OAMG;IACH,MAAM,CAAC,IAAY;QACjB,OAAO,IAAI,WAAW,CAAC,IAAI,CAAC,CAAC;IAC/B,CAAC;IAED;;;;;;;;OAQG;IACH,QAAQ,CAAC,OAA+B;QACtC,MAAM,QAAQ,GAAmB,EAAE,CAAC;QACpC,MAAM,KAAK,GAAG,IAAI,GAAG,EAAU,CAAC;QAChC,MAAM,MAAM,GAAG,IAAI,GAAG,EAAU,CAAC;QACjC,MAAM,QAAQ,GAAmB,EAAE,CAAC;QACpC,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;YAC7B,IAAI,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;gBAC3B,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,cAAc,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,mBAAmB,EAAE,CAAC,CAAC,CAAC;YAC7G,CAAC;YACD,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;YACvB,KAAK,MAAM,OAAO,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;gBACtC,IAAI,MAAM,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC;oBAAE,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,iBAAiB,EAAE,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;gBACnG,MAAM,CAAC,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;gBAC1B,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC;YACzB,CAAC;QACH,CAAC;QACD,MAAM,CAAC,QAAQ,CAAC,CAAC;QACjB,OAAO,EAAE,OAAO,EAAE,CAAC,GAAG,KAAK,CAAC,EAAE,QAAQ,EAAE,CAAC;IAC3C,CAAC;CACF,CAAC"}

package/dist/src/check/index.d.ts

@@ -0,0 +1,15 @@
+/**
+ * Copyright (c) 2025 Elara AI Pty Ltd
+ * Licensed under BSL 1.1. See LICENSE for details.
+ */
+/**
+ * `@elaraai/e3-cloud-files/check`: the checks a deploy of declarations and a
+ * bind run, and the keys, prefixes, globs and URLs they hold to, without the
+ * authoring (which needs `@elaraai/e3`): what the CLI and the platform
+ * import.
+ *
+ * @packageDocumentation
+ */
+export { checkFiles, checkBindings, readPackageInputs, shapeType, type PackageInput, type ObjectReader, type BindingPolicy, } from '../check.js';
+export { globMatcher, parseBucketUrl, keyProblem, prefixProblem, includeProblem, type BucketLocation } from '../paths.js';
+//# sourceMappingURL=index.d.ts.map

package/dist/src/check/index.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/check/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;GAOG;AAEH,OAAO,EACL,UAAU,EACV,aAAa,EACb,iBAAiB,EACjB,SAAS,EACT,KAAK,YAAY,EACjB,KAAK,YAAY,EACjB,KAAK,aAAa,GACnB,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,WAAW,EAAE,cAAc,EAAE,UAAU,EAAE,aAAa,EAAE,cAAc,EAAE,KAAK,cAAc,EAAE,MAAM,aAAa,CAAC"}