@elaraai/e3-cloud-files 1.0.94 → 1.0.97

package/README.md

@@ -9,8 +9,17 @@ changed, the same way on a local repository and on a deployed workspace.
 - **Your data stays in the bucket.** Locally, a sync reads the bucket with
   your own cloud credentials. The bytes land only in the local e3 repository,
   never in your project.
-- **A file arrives as it is**, a `Blob`. A task decodes it in East: as East's
-  JSON, as CSV, or as text.
+- **A file arrives as it is**, a `Blob`, and a task decodes it in East: as
+  East's JSON, as CSV, or as text. Or, named with a **type**, a beast2 file
+  arrives as the value it holds, an input of that type with no task in
+  between, a collection of any size.
+- **Records are seeded and fed from files.** A record's mutation may be fed
+  by files: each new file is the mutation's one argument, applied once.
+  (A record seeded from a file, `upload.record`, lands with the next e3
+  release: see below.)
+- **Push from disk.** `e3-cloud files push` puts files from your machine
+  into a source and takes them in, with your e3 login alone where the source
+  is the deployment's own bucket.
 - **Nothing polls.** A sync runs when someone asks for it, and reads only the
   files that changed since the last one.
 
@@ -54,9 +63,12 @@ export default Files.manifest([upload]);
 | Shape | Selects | The input's type | Largest file |
 |---|---|---|---|
 | `file(input, { key })` | One key | `Blob` | 1 GiB less 64 KiB |
+| `file(input, { key, type })` | One key, a beast2 file of `type` | `type` | 1 GiB for a value that is no collection; a collection any size (a segment at most 64 MiB) |
 | `latest(input, { prefix, include? })` | The greatest key under the prefix that matches | `Blob` | 1 GiB less 64 KiB |
+| `latest(input, { prefix, include?, type })` | The greatest key, a beast2 file of `type` | `type` | as a typed `file` |
 | `folder(input, { prefix, include? })` | Every key under the prefix that matches, by its path under the prefix | `Dict<String, Blob>` | 64 MiB less 64 KiB each |
 | `listing(input, { prefix, include? })` | The same keys' size, time and ETag, without their bytes | `Dict<String, Struct{size, modified, etag}>` | none |
+| `mutation(def, { key } \| { prefix, include? })` | One key, or every file under a prefix its glob matches, each new one in key order: a beast2 file of the mutation's one argument's type | (the mutation's, as declared) | 1 GiB |
 
 - **Each shape returns an ordinary `e3.input`.** It has no default, so it
   stays unassigned until a sync sets it. A task takes it like any input, and
@@ -72,8 +84,60 @@ export default Files.manifest([upload]);
   subfolders' too, their paths naming the subfolder (`2026/10/a.json`).
 - **Mistakes are refused as they are declared** (`FilesDeclarationError`, one
   line per problem): a name e3 refuses, a key that isn't relative, a glob
-  that can't match, and in `Files.manifest` two sources of one name or two
-  shapes filling one input.
+  that can't match, a `patch` mutation or one of another arity than one, and
+  in `Files.manifest` two sources of one name or two shapes filling one
+  target.
+
+## Typed files and fed records
+
+A beast2 file already is an East value: the East SDK (`encodeBeast2For`,
+`Beast2ElementWriter`), east-py, `e3 convert` and `e3 dataset get -f beast2`
+all write it. A `file` or `latest` that names a `type` is an input of that
+type, and the sync takes the file in as the value it holds, with no task in
+between:
+
+```ts
+// src/package.ts
+export const Price = StructType({ sku: StringType, price: FloatType });
+export const prices = upload.file('prices', { key: 'prices.beast2', type: ArrayType(Price) });   // an input of Array<Price>
+export const stock  = upload.latest('stock', { prefix: 'stock/', type: ArrayType(Stock) });       // the greatest key, typed
+
+export const priceBook  = e3.record('price_book', DictType(StringType, FloatType), new Map());
+export const loadPrices = e3.mutation.reduce('load_prices', priceBook, East.function([PriceBook, ArrayType(Price)], PriceBook, …));
+export const summary = e3.task('price_summary', [prices, priceBook], …);   // used as any dataset
+
+// src/files.ts
+upload.mutation(loadPrices, { prefix: 'deltas/', include: '*.beast2' });   // each new delta file calls load_prices with its value
+export default Files.manifest([upload]);
+```
+
+- **The header first.** The sync reads a typed file's beast2 header before
+  any other byte of it and refuses a file of another type (`type_mismatch`,
+  naming both types) or no beast2 (`not_beast2`); the input keeps its value
+  and the sync goes on. A deploy of the declarations checks the type against
+  the package's input, record or mutation too.
+- **Any size.** A collection (an `Array`, a `Set`, a `Dict`) is taken in by
+  pieces, as e3 takes in any delivery, however large; in the cloud never
+  through a disk. A value that is no collection is held whole: at most 1 GiB.
+  A file whose segments are larger than beast2's 64 MiB is refused naming
+  the fix (write it again with a current writer).
+- **The same bytes again change nothing.** A file re-uploaded unchanged (a
+  new ETag, the same SHA-256) gives the same value: nothing downstream runs.
+- **A fed mutation** (`upload.mutation(def, …)`) is the package's own, a
+  `reduce` or an `edit` taking exactly one argument (a `patch` cannot be
+  fed). Every new file the mapping selects is applied, in key order, once by
+  its bytes' hash: a sync run again applies nothing twice, and the same bytes
+  under another name are applied no more; a file the mutation refuses
+  (`invalid`, `failed`, `timed_out`) is reported and tried again only once it
+  changes, and the files after it wait; a feed of a record with no state yet,
+  or one busy, waits for the next sync. Its commit reads in the record's
+  history as any mutation's, its argument the delivery.
+- **A seeded record** (`upload.record(name, type, { key })`, a record pending
+  until its file arrives, which is its first state) needs the e3 release
+  that lets a record be declared with no initial value
+  (elaraai/east-workspace#1363): the declarations, the checks (`record`
+  targets, `$init` and `$seed` commits, `--reset`) and the sync's seeds are
+  in place, and the declaration itself lands with that release.
 
 ## Decoding in a task
 
@@ -138,13 +202,38 @@ e3 dataflow run ./repo dev
   only each folder's n greatest keys, which caps what comes down to your
   machine.
 - **`files sync`** lists each source and compares every file with what its
-  input took last, by ETag and size. It reads only what is new or changed,
-  writes only the inputs whose files changed, and says what it did per input.
-  `--input` syncs only those inputs; `--dry-run` lists what a sync would take
-  and takes nothing. A sync doesn't run the dataflow: `e3 dataflow run` does, and the
-  sync prints the command when anything changed.
-- **`files status`** shows the declarations, where each source is bound, and
-  the last sync.
+  target took last, by ETag and size. It reads only what is new or changed,
+  writes only the targets whose files changed, and says what it did per
+  target: the rows taken in, the commit a feed made, what was refused and
+  why. `--target` syncs only those (an input's name, `record:<name>`,
+  `mutation:<record>.<name>`, split at its last dot, so a mutation whose
+  name holds a dot is named only through the API's structured targets);
+  `--reset <record>` takes a seeded record's
+  file as its state again whatever has mutated it since; `--dry-run` lists
+  what a sync would take and takes nothing. A sync doesn't run the dataflow:
+  `e3 dataflow run` does, and the sync prints the command when anything
+  changed.
+- **`files push`** puts files from disk into a source and takes them in:
+  each file is matched to the mapping that takes it (a `file` by its name, a
+  `latest` under its prefix, a directory to the `folder`; or `--as <target>`
+  names one for every file), checked against the mapping's type and limit,
+  put where the source is bound (with your e3 login alone where that is the
+  deployment's own bucket: the platform signs the upload, bound to the
+  file's SHA-256; with your own credentials, `--profile`, elsewhere), then
+  one sync of what was put. A file the source holds already is `unchanged`;
+  `--dry-run` plans and puts nothing. Running it again puts only what is
+  not there.
+- **`files status`** shows each mapping with its shape, its type, the file
+  it holds and the commit it made (or `pending` for a record not yet
+  seeded), where each source is bound, and the last sync.
+
+```
+$ e3-cloud files push https://e3.example.com/repos/acme dev ./data/prices.beast2 ./data/deltas/2026-10-19.beast2
+upload  s3://acme-files/acme/dev/
+  prices.beast2                   → prices                         Array<Struct{sku, price}>  38 MiB  put  taken in  412,310 rows
+  2026-10-19.beast2               → mutation:price_book.load_prices  load_prices               1 MiB  put  committed 9f2c1a0b7e3d
+Next: e3 dataflow run https://e3.example.com/repos/acme dev
+```
 
 A sync is safe to repeat and to stop. Each input is remembered only once it
 is written, so a sync that stops part way takes the rest next time.
@@ -229,8 +318,11 @@ is read. So a sync holds one batch of files and the pieces it changes,
 however large the folder; the first sync of 6 GB of files holds 256 MiB at a
 time, and a sync stopped part way takes the rest next time.
 
-A `file` or `latest` input is held whole while it is written: a 1 GiB file
-needs some 3 GiB of memory.
+A `file` or `latest` input with no type is held whole while it is written: a
+1 GiB file needs some 3 GiB of memory. A typed collection is never held
+whole: in the cloud it is staged in the data bucket as it streams and taken
+in by intake units on the sandbox functions; locally it is streamed to the
+repository's own scratch and taken in by e3's door for a file.
 
 ### On a local repository
 
@@ -248,8 +340,11 @@ and profile it was: `aws sso login --profile <name>` renews it.
 
 `@elaraai/e3-cloud-files/check` holds the checks without the authoring:
 
-- `checkFiles(manifest, inputs)`: the declarations against a package's inputs
-  (`readPackageInputs(read, packageHash)` reads them from its objects);
+- `checkFiles(manifest, surface)`: the declarations against a package's
+  inputs, records and mutations (`readPackageSurface(read, packageHash)`
+  reads them from its objects): an input writable and of the mapping's type,
+  a record of the file's type, a mutation of the record taking one argument
+  of the file's type;
 - `checkBindings(manifest, bindings, policy)`: bindings against the
   declarations, and in the cloud against the sources the deployment grants.
 
@@ -260,13 +355,23 @@ rendered as one line by `filesProblemText`.
 
 In `@elaraai/e3-cloud-types`, re-exported here:
 
-- `FilesManifestType`: the sources' names, and each mapping's input, source
-  and shape (`FilesShapeType`: `file`, `latest`, `folder`, `listing`).
+- `FilesManifestType`: the sources' names, and each mapping's target
+  (`FilesTargetType`: `input`, `record` or `mutation`), source and shape
+  (`FilesShapeType`: `file` and `latest`, each with an optional `type`;
+  `folder`, `listing`).
 - `FilesBindingType`: a source's URL, region, and, locally, profile and
   `latest`.
-- `FilesSyncReportType`: per input, the files new, changed, removed,
-  unchanged and refused, the bytes taken in, and whether it was written.
-- `FilesStatusType`, `FilesProblemType`.
+- `FilesSyncReportType`: per target, the files new, changed, removed,
+  unchanged and refused (`FilesRefusalReasonType`: `too_large`,
+  `segment_too_large`, `unreadable`, `not_beast2`, `type_mismatch`,
+  `mutation`, `missing`), the bytes taken in, whether it was written, the
+  typed value taken (`FilesTakenType`: its rows), the commit a seed or feed
+  made, and why a seed was kept or a feed waits.
+- `FilesStatusType` (each mapping with what it holds and whether its record
+  is pending), `FilesProblemType`, the push's `FilesPushRequestType` and
+  `FilesPushPlanType`.
+- Values stored by the first version (mappings by input) are read through
+  `@elaraai/e3-cloud-files/compat`, which lifts them.
 - The inputs' types: `FilesFileType` (`Blob`), `FilesFolderType`
   (`Dict<String, Blob>`), `FilesListingType`.
 
@@ -276,10 +381,11 @@ In `@elaraai/e3-cloud-types`, re-exported here:
 |---|---|
 | `@elaraai/e3-cloud-files` | Authoring: `Files.source`, the shapes, `Files.manifest`, and the types |
 | `@elaraai/e3-cloud-files/check` | The checks, without the authoring |
-| `@elaraai/e3-cloud-files/sync` | The sync (`syncFiles`) over what reads, writes and remembers, and those three in memory for tests |
-| `@elaraai/e3-cloud-files/storage` | The sync's writer over an e3 store (needs `@elaraai/e3-core`) |
+| `@elaraai/e3-cloud-files/sync` | The sync (`syncFiles`) over what reads, writes and remembers, the push's plan (`planPush`), and the ports in memory for tests |
+| `@elaraai/e3-cloud-files/storage` | The sync's writer over an e3 store, and the cloud's over a staging (needs `@elaraai/e3-core`) |
 | `@elaraai/e3-cloud-files/s3` | The sync's reader of a bucket (needs `@aws-sdk/client-s3`) |
-| `@elaraai/e3-cloud-files/examples/*` | The worked example (`orders`) and the decoding examples (`decode`) |
+| `@elaraai/e3-cloud-files/compat` | The first version's stored types, and their lift into the current ones |
+| `@elaraai/e3-cloud-files/examples/*` | The worked examples (`orders`, `typed`) and the decoding examples (`decode`) |
 
 ## License
 

package/dist/src/authoring.d.ts

@@ -3,29 +3,42 @@
  * Licensed under BSL 1.1. See LICENSE for details.
  */
 /**
- * Declaring which inputs come from files.
+ * Declaring which inputs come from files, and which records are seeded or
+ * fed from them.
  *
  * ```ts
  * export const upload = Files.source('upload');
- * export const rates  = upload.file('rates', { key: 'rates.json' });
- * export const stock  = upload.latest('stock', { prefix: 'stock/' });
- * export const orders = upload.folder('orders', { prefix: 'orders/', include: '*.json' });
+ * export const rates  = upload.file('rates', { key: 'rates.json' });                           // a Blob
+ * export const orders = upload.file('orders', { key: 'orders.beast2', type: ArrayType(Order) }); // an Array<Order>
+ * export const stock  = upload.latest('stock', { prefix: 'stock/', type: ArrayType(Stock) });
+ * export const files  = upload.folder('invoices', { prefix: 'invoices/', include: '*.pdf' });
+ * upload.mutation(loadPrices, { key: 'prices.beast2' });                                        // each new file calls loadPrices
  * export default Files.manifest([upload]);
  * ```
  *
- * Each declaration is an ordinary `e3.input` with no default, so the package
- * is plain e3 and tasks take these inputs as any other. A sync fills them;
- * until the first, they are unassigned. What each source's files fill is the
- * {@link Files.manifest}, which `e3-cloud files deploy` deploys beside the
- * package.
+ * Each input declaration is an ordinary `e3.input` with no default, so the
+ * package is plain e3 and tasks take these inputs as any other. A sync fills
+ * them; until the first, they are unassigned. A `file` or `latest` that
+ * names a `type` is an input of that type, filled with the beast2 value the
+ * file holds; one that names none is a `Blob`, filled with the file's
+ * bytes. A mutation declared here is the package's own (`e3.mutation.reduce`
+ * or `edit`, taking one argument): a sync calls it once with each new file's
+ * value. What each source's files fill is the {@link Files.manifest}, which
+ * `e3-cloud files deploy` deploys beside the package.
  *
  * @packageDocumentation
  */
-import e3 from '@elaraai/e3';
+import e3, { type MutationDef } from '@elaraai/e3';
 import { type EastType } from '@elaraai/east';
 import { FilesFileType, FilesFolderType, FilesListingType, type FilesManifest, type FilesMapping, type FilesProblem } from '@elaraai/e3-cloud-types';
-/** The input a declaration makes: an `e3.input` of its shape's type, with no default. */
+/** The input a declaration makes: an `e3.input` of its type, with no default. */
 export type FilesInput<Name extends string, T extends EastType> = ReturnType<typeof e3.input<Name, T>>;
+/** The mutation a feed declaration hands back: the package's own definition, as it was given. */
+export type FilesMutation<M extends MutationDef> = M;
+/** One file, by its key relative to where the workspace binds the source. */
+export interface FilesKeySpec {
+    readonly key: string;
+}
 /** The selection of `latest`, `folder` and `listing`: a prefix, and a glob over the paths under it. */
 export interface FilesSelection {
     /** The folder, relative to where the workspace binds the source: `''` for all of it, else ending with `/` */
@@ -33,6 +46,14 @@ export interface FilesSelection {
     /** A glob over paths relative to the prefix: `*` within a segment, `**` across segments, `?` one character, `[a-z]` one of a set, `{csv,json}` any of its alternatives */
     readonly include?: string;
 }
+/** One file, typed: the beast2 value of `type` it holds. */
+export interface FilesTypedKeySpec<T extends EastType> extends FilesKeySpec {
+    readonly type: T;
+}
+/** The newest file of a selection, typed. */
+export interface FilesTypedSelection<T extends EastType> extends FilesSelection {
+    readonly type: T;
+}
 /** Thrown when declarations are not well-formed, naming every problem. */
 export declare class FilesDeclarationError extends Error {
     /** Every problem */
@@ -41,7 +62,7 @@ export declare class FilesDeclarationError extends Error {
 }
 /**
  * A source of files: a name each workspace binds to a bucket and a prefix,
- * and the inputs its files fill.
+ * and the inputs, records and mutations its files fill.
  */
 export declare class FilesSource {
     #private;
@@ -49,27 +70,30 @@ export declare class FilesSource {
     readonly name: string;
     /** @internal Use {@link Files.source}. */
     constructor(name: string);
-    /** The inputs this source fills, in the order they were declared. */
+    /** The targets this source fills, in the order they were declared. */
     get mappings(): readonly FilesMapping[];
     /**
-     * One file, as a `Blob`: the input `.inputs.<input>`.
+     * One file: the input `.inputs.<input>`. As a `Blob`, its bytes as they
+     * are; or, with `type`, as the beast2 value of that type the file holds
+     * (a collection of any size, taken in by pieces).
      *
      * @param input - The input's name
-     * @param spec - `key`: the file, relative to where the workspace binds the source
+     * @param spec - `key`: the file, relative to where the workspace binds the source; `type`: the value it holds
      * @returns The input, for tasks to take
      */
-    file<Name extends string>(input: Name, spec: {
-        readonly key: string;
-    }): FilesInput<Name, typeof FilesFileType>;
+    file<Name extends string>(input: Name, spec: FilesKeySpec): FilesInput<Name, typeof FilesFileType>;
+    file<Name extends string, T extends EastType>(input: Name, spec: FilesTypedKeySpec<T>): FilesInput<Name, T>;
     /**
-     * The newest file under a prefix, as a `Blob`: the greatest key that matches
-     * `include`, so files named by their time pick the latest.
+     * The newest file under a prefix: the greatest key that matches
+     * `include`, so files named by their time pick the latest. As a `Blob`,
+     * or, with `type`, as the value it holds.
      *
      * @param input - The input's name
-     * @param spec - The prefix, and a glob over the paths under it
+     * @param spec - The prefix, a glob over the paths under it, and the value's type
      * @returns The input, for tasks to take
      */
     latest<Name extends string>(input: Name, spec: FilesSelection): FilesInput<Name, typeof FilesFileType>;
+    latest<Name extends string, T extends EastType>(input: Name, spec: FilesTypedSelection<T>): FilesInput<Name, T>;
     /**
      * Every file under a prefix that matches `include`, as a
      * `Dict<String, Blob>` keyed by each file's path relative to the prefix.
@@ -89,6 +113,19 @@ export declare class FilesSource {
      * @returns The input, for tasks to take
      */
     listing<Name extends string>(input: Name, spec: FilesSelection): FilesInput<Name, typeof FilesListingType>;
+    /**
+     * A record's mutation, fed: a sync calls it once with each new file's
+     * value, which must be of the mutation's one argument's type. The
+     * mutation is the package's own, a `reduce` or an `edit` taking exactly
+     * one argument; a `patch` mutation takes a patch, not a value, so it
+     * cannot be fed.
+     *
+     * @param def - The mutation: `e3.mutation.reduce(...)` or `e3.mutation.edit(...)`
+     * @param spec - One file (`key`), or the newest under a prefix (`prefix`, `include`)
+     * @returns The mutation, as given
+     * @throws {FilesDeclarationError} When the mutation takes other than one argument, or is a `patch`
+     */
+    mutation<M extends MutationDef>(def: M, spec: FilesKeySpec | FilesSelection): FilesMutation<M>;
 }
 /** Declaring files. */
 export declare const Files: {
@@ -96,18 +133,18 @@ export declare const Files: {
      * A source of files, which each workspace binds to a bucket and a prefix.
      *
      * @param name - Its name, unique among a solution's sources
-     * @returns The source, whose methods declare the inputs it fills
+     * @returns The source, whose methods declare what its files fill
      * @throws {FilesDeclarationError} When the name is one e3 refuses
      */
     source(name: string): FilesSource;
     /**
      * The declarations `e3-cloud files deploy` deploys: the sources, and the
-     * inputs their files fill.
+     * inputs, records and mutations their files fill.
      *
      * @param sources - Every source
      * @returns The manifest, an East value of `FilesManifestType`
      * @throws {FilesDeclarationError} When two sources share a name, or two
-     *   mappings fill one input
+     *   mappings fill one target
      */
     manifest(sources: readonly FilesSource[]): FilesManifest;
 };

package/dist/src/authoring.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"authoring.d.ts","sourceRoot":"","sources":["../../src/authoring.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;GAkBG;AAEH,OAAO,EAAE,MAAM,aAAa,CAAC;AAC7B,OAAO,EAAuB,KAAK,QAAQ,EAAE,MAAM,eAAe,CAAC;AAEnE,OAAO,EACL,aAAa,EAAE,eAAe,EAAE,gBAAgB,EAEhD,KAAK,aAAa,EAAE,KAAK,YAAY,EAAE,KAAK,YAAY,EACzD,MAAM,yBAAyB,CAAC;AAGjC,yFAAyF;AACzF,MAAM,MAAM,UAAU,CAAC,IAAI,SAAS,MAAM,EAAE,CAAC,SAAS,QAAQ,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;AAEvG,uGAAuG;AACvG,MAAM,WAAW,cAAc;IAC7B,6GAA6G;IAC7G,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,0KAA0K;IAC1K,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED,0EAA0E;AAC1E,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,oBAAoB;IACpB,QAAQ,CAAC,QAAQ,EAAE,SAAS,YAAY,EAAE,CAAC;gBAE/B,QAAQ,EAAE,SAAS,YAAY,EAAE;CAK9C;AA8BD;;;GAGG;AACH,qBAAa,WAAW;;IACtB,wBAAwB;IACxB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAGtB,0CAA0C;gBAC9B,IAAI,EAAE,MAAM;IAKxB,qEAAqE;IACrE,IAAI,QAAQ,IAAI,SAAS,YAAY,EAAE,CAEtC;IASD;;;;;;OAMG;IACH,IAAI,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE;QAAE,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAA;KAAE,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAM9G;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAItG;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,eAAe,CAAC;IAIxG;;;;;;;;OAQG;IACH,OAAO,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,gBAAgB,CAAC;CAG3G;AAED,uBAAuB;AACvB,eAAO,MAAM,KAAK;IAChB;;;;;;OAMG;iBACU,MAAM,GAAG,WAAW;IAIjC;;;;;;;;OAQG;sBACe,SAAS,WAAW,EAAE,GAAG,aAAa;CAmBzD,CAAC"}
+{"version":3,"file":"authoring.d.ts","sourceRoot":"","sources":["../../src/authoring.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;;;;;;;;GAyBG;AAEH,OAAO,EAAE,EAAE,EAAE,KAAK,WAAW,EAAE,MAAM,aAAa,CAAC;AACnD,OAAO,EAAwC,KAAK,QAAQ,EAAE,MAAM,eAAe,CAAC;AAEpF,OAAO,EACL,aAAa,EAAE,eAAe,EAAE,gBAAgB,EAEhD,KAAK,aAAa,EAAE,KAAK,YAAY,EAAE,KAAK,YAAY,EACzD,MAAM,yBAAyB,CAAC;AAGjC,iFAAiF;AACjF,MAAM,MAAM,UAAU,CAAC,IAAI,SAAS,MAAM,EAAE,CAAC,SAAS,QAAQ,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;AAEvG,iGAAiG;AACjG,MAAM,MAAM,aAAa,CAAC,CAAC,SAAS,WAAW,IAAI,CAAC,CAAC;AAErD,6EAA6E;AAC7E,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAC;CACtB;AAED,uGAAuG;AACvG,MAAM,WAAW,cAAc;IAC7B,6GAA6G;IAC7G,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,0KAA0K;IAC1K,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED,4DAA4D;AAC5D,MAAM,WAAW,iBAAiB,CAAC,CAAC,SAAS,QAAQ,CAAE,SAAQ,YAAY;IACzE,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC;CAClB;AAED,6CAA6C;AAC7C,MAAM,WAAW,mBAAmB,CAAC,CAAC,SAAS,QAAQ,CAAE,SAAQ,cAAc;IAC7E,QAAQ,CAAC,IAAI,EAAE,CAAC,CAAC;CAClB;AAED,0EAA0E;AAC1E,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,oBAAoB;IACpB,QAAQ,CAAC,QAAQ,EAAE,SAAS,YAAY,EAAE,CAAC;gBAE/B,QAAQ,EAAE,SAAS,YAAY,EAAE;CAK9C;AAmDD;;;GAGG;AACH,qBAAa,WAAW;;IACtB,wBAAwB;IACxB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAGtB,0CAA0C;gBAC9B,IAAI,EAAE,MAAM;IAKxB,sEAAsE;IACtE,IAAI,QAAQ,IAAI,SAAS,YAAY,EAAE,CAEtC;IAcD;;;;;;;;OAQG;IACH,IAAI,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,YAAY,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAClG,IAAI,CAAC,IAAI,SAAS,MAAM,EAAE,CAAC,SAAS,QAAQ,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,iBAAiB,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,IAAI,EAAE,CAAC,CAAC;IAQ3G;;;;;;;;OAQG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IACtG,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,CAAC,SAAS,QAAQ,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,mBAAmB,CAAC,CAAC,CAAC,GAAG,UAAU,CAAC,IAAI,EAAE,CAAC,CAAC;IAQ/G;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,eAAe,CAAC;IAIxG;;;;;;;;OAQG;IACH,OAAO,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,gBAAgB,CAAC;IAI1G;;;;;;;;;;;OAWG;IACH,QAAQ,CAAC,CAAC,SAAS,WAAW,EAAE,GAAG,EAAE,CAAC,EAAE,IAAI,EAAE,YAAY,GAAG,cAAc,GAAG,aAAa,CAAC,CAAC,CAAC;CAW/F;AAED,uBAAuB;AACvB,eAAO,MAAM,KAAK;IAChB;;;;;;OAMG;iBACU,MAAM,GAAG,WAAW;IAIjC;;;;;;;;OAQG;sBACe,SAAS,WAAW,EAAE,GAAG,aAAa;CAmBzD,CAAC"}

package/dist/src/authoring.js

@@ -3,28 +3,35 @@
  * Licensed under BSL 1.1. See LICENSE for details.
  */
 /**
- * Declaring which inputs come from files.
+ * Declaring which inputs come from files, and which records are seeded or
+ * fed from them.
  *
  * ```ts
  * export const upload = Files.source('upload');
- * export const rates  = upload.file('rates', { key: 'rates.json' });
- * export const stock  = upload.latest('stock', { prefix: 'stock/' });
- * export const orders = upload.folder('orders', { prefix: 'orders/', include: '*.json' });
+ * export const rates  = upload.file('rates', { key: 'rates.json' });                           // a Blob
+ * export const orders = upload.file('orders', { key: 'orders.beast2', type: ArrayType(Order) }); // an Array<Order>
+ * export const stock  = upload.latest('stock', { prefix: 'stock/', type: ArrayType(Stock) });
+ * export const files  = upload.folder('invoices', { prefix: 'invoices/', include: '*.pdf' });
+ * upload.mutation(loadPrices, { key: 'prices.beast2' });                                        // each new file calls loadPrices
  * export default Files.manifest([upload]);
  * ```
  *
- * Each declaration is an ordinary `e3.input` with no default, so the package
- * is plain e3 and tasks take these inputs as any other. A sync fills them;
- * until the first, they are unassigned. What each source's files fill is the
- * {@link Files.manifest}, which `e3-cloud files deploy` deploys beside the
- * package.
+ * Each input declaration is an ordinary `e3.input` with no default, so the
+ * package is plain e3 and tasks take these inputs as any other. A sync fills
+ * them; until the first, they are unassigned. A `file` or `latest` that
+ * names a `type` is an input of that type, filled with the beast2 value the
+ * file holds; one that names none is a `Blob`, filled with the file's
+ * bytes. A mutation declared here is the package's own (`e3.mutation.reduce`
+ * or `edit`, taking one argument): a sync calls it once with each new file's
+ * value. What each source's files fill is the {@link Files.manifest}, which
+ * `e3-cloud files deploy` deploys beside the package.
  *
  * @packageDocumentation
  */
 import e3 from '@elaraai/e3';
-import { none, some, variant } from '@elaraai/east';
+import { none, some, toEastTypeValue, variant } from '@elaraai/east';
 import { nameProblem } from '@elaraai/e3-types';
-import { FilesFileType, FilesFolderType, FilesListingType, filesProblemText, } from '@elaraai/e3-cloud-types';
+import { FilesFileType, FilesFolderType, FilesListingType, filesProblemText, sameFilesTarget, } from '@elaraai/e3-cloud-types';
 import { includeProblem, keyProblem, prefixProblem } from './paths.js';
 /** Thrown when declarations are not well-formed, naming every problem. */
 export class FilesDeclarationError extends Error {
@@ -46,16 +53,21 @@ function nameProblems(kind, name) {
     const reason = nameProblem('workspace', name);
     return reason === null ? [] : [variant('invalid_name', { kind, name, reason })];
 }
+/** A key's problems. */
+function keyProblems(target, key) {
+    const reason = keyProblem(key);
+    return reason === null ? [] : [variant('invalid_key', { target, key, reason })];
+}
 /** A selection's problems. */
-function selectionProblems(input, selection) {
+function selectionProblems(target, selection) {
     const problems = [];
     const prefix = prefixProblem(selection.prefix);
     if (prefix !== null)
-        problems.push(variant('invalid_key', { input, key: selection.prefix, reason: prefix }));
+        problems.push(variant('invalid_key', { target, key: selection.prefix, reason: prefix }));
     if (selection.include !== undefined) {
         const include = includeProblem(selection.include);
         if (include !== null)
-            problems.push(variant('invalid_include', { input, include: selection.include, reason: include }));
+            problems.push(variant('invalid_include', { target, include: selection.include, reason: include }));
     }
     return problems;
 }
@@ -63,9 +75,23 @@ function selectionProblems(input, selection) {
 function selected(selection) {
     return { prefix: selection.prefix, include: selection.include === undefined ? none : some(selection.include) };
 }
+/** Whether a spec names one file, rather than a selection. */
+function isKeySpec(spec) {
+    return 'key' in spec;
+}
+/**
+ * The shape of a spec that selects one file, typed: `file` or `latest`, with
+ * the type, and its problems.
+ */
+function oneFileShape(target, spec, type) {
+    const typeValue = some(toEastTypeValue(type));
+    if (isKeySpec(spec))
+        return { shape: variant('file', { key: spec.key, type: typeValue }), problems: keyProblems(target, spec.key) };
+    return { shape: variant('latest', { ...selected(spec), type: typeValue }), problems: selectionProblems(target, spec) };
+}
 /**
  * A source of files: a name each workspace binds to a bucket and a prefix,
- * and the inputs its files fill.
+ * and the inputs, records and mutations its files fill.
  */
 export class FilesSource {
     /** The source's name */
@@ -76,38 +102,31 @@ export class FilesSource {
         refuse(nameProblems('source', name));
         this.name = name;
     }
-    /** The inputs this source fills, in the order they were declared. */
+    /** The targets this source fills, in the order they were declared. */
     get mappings() {
         return this.#mappings;
     }
-    /** Records a mapping and makes its input. */
-    #declare(input, type, shape, problems) {
-        refuse([...nameProblems('input', input), ...problems]);
-        this.#mappings.push({ input, source: this.name, shape });
+    /** Records a mapping, refusing its problems first. */
+    #declare(target, shape, problems) {
+        refuse(problems);
+        this.#mappings.push({ target, source: this.name, shape });
+    }
+    /** Records an input's mapping and makes its input. */
+    #input(input, type, shape, problems) {
+        this.#declare(variant('input', input), shape, [...nameProblems('input', input), ...problems]);
         return e3.input(input, type);
     }
-    /**
-     * One file, as a `Blob`: the input `.inputs.<input>`.
-     *
-     * @param input - The input's name
-     * @param spec - `key`: the file, relative to where the workspace binds the source
-     * @returns The input, for tasks to take
-     */
     file(input, spec) {
-        const reason = keyProblem(spec.key);
-        const problems = reason === null ? [] : [variant('invalid_key', { input, key: spec.key, reason })];
-        return this.#declare(input, FilesFileType, variant('file', { key: spec.key }), problems);
+        const target = variant('input', input);
+        const type = 'type' in spec ? spec.type : FilesFileType;
+        const shape = variant('file', { key: spec.key, type: 'type' in spec ? some(toEastTypeValue(spec.type)) : none });
+        return this.#input(input, type, shape, keyProblems(target, spec.key));
     }
-    /**
-     * The newest file under a prefix, as a `Blob`: the greatest key that matches
-     * `include`, so files named by their time pick the latest.
-     *
-     * @param input - The input's name
-     * @param spec - The prefix, and a glob over the paths under it
-     * @returns The input, for tasks to take
-     */
     latest(input, spec) {
-        return this.#declare(input, FilesFileType, variant('latest', selected(spec)), selectionProblems(input, spec));
+        const target = variant('input', input);
+        const type = 'type' in spec ? spec.type : FilesFileType;
+        const shape = variant('latest', { ...selected(spec), type: 'type' in spec ? some(toEastTypeValue(spec.type)) : none });
+        return this.#input(input, type, shape, selectionProblems(target, spec));
     }
     /**
      * Every file under a prefix that matches `include`, as a
@@ -118,7 +137,7 @@ export class FilesSource {
      * @returns The input, for tasks to take
      */
     folder(input, spec) {
-        return this.#declare(input, FilesFolderType, variant('folder', selected(spec)), selectionProblems(input, spec));
+        return this.#input(input, FilesFolderType, variant('folder', selected(spec)), selectionProblems(variant('input', input), spec));
     }
     /**
      * The files under a prefix that match `include`, without their bytes: each
@@ -130,7 +149,32 @@ export class FilesSource {
      * @returns The input, for tasks to take
      */
     listing(input, spec) {
-        return this.#declare(input, FilesListingType, variant('listing', selected(spec)), selectionProblems(input, spec));
+        return this.#input(input, FilesListingType, variant('listing', selected(spec)), selectionProblems(variant('input', input), spec));
+    }
+    /**
+     * A record's mutation, fed: a sync calls it once with each new file's
+     * value, which must be of the mutation's one argument's type. The
+     * mutation is the package's own, a `reduce` or an `edit` taking exactly
+     * one argument; a `patch` mutation takes a patch, not a value, so it
+     * cannot be fed.
+     *
+     * @param def - The mutation: `e3.mutation.reduce(...)` or `e3.mutation.edit(...)`
+     * @param spec - One file (`key`), or the newest under a prefix (`prefix`, `include`)
+     * @returns The mutation, as given
+     * @throws {FilesDeclarationError} When the mutation takes other than one argument, or is a `patch`
+     */
+    mutation(def, spec) {
+        const record = def.record.name;
+        const target = variant('mutation', { record, mutation: def.name });
+        const problems = [...nameProblems('record', record), ...nameProblems('mutation', def.name)];
+        if (def.form === 'patch')
+            problems.push(variant('mutation_patch', { record, mutation: def.name }));
+        else if (def.argTypes.length !== 1)
+            problems.push(variant('mutation_arity', { record, mutation: def.name, arity: BigInt(def.argTypes.length) }));
+        refuse(problems);
+        const { shape, problems: specProblems } = oneFileShape(target, spec, def.argTypes[0]);
+        this.#declare(target, shape, specProblems);
+        return def;
     }
 }
 /** Declaring files. */
@@ -139,7 +183,7 @@ export const Files = {
      * A source of files, which each workspace binds to a bucket and a prefix.
      *
      * @param name - Its name, unique among a solution's sources
-     * @returns The source, whose methods declare the inputs it fills
+     * @returns The source, whose methods declare what its files fill
      * @throws {FilesDeclarationError} When the name is one e3 refuses
      */
     source(name) {
@@ -147,17 +191,17 @@ export const Files = {
     },
     /**
      * The declarations `e3-cloud files deploy` deploys: the sources, and the
-     * inputs their files fill.
+     * inputs, records and mutations their files fill.
      *
      * @param sources - Every source
      * @returns The manifest, an East value of `FilesManifestType`
      * @throws {FilesDeclarationError} When two sources share a name, or two
-     *   mappings fill one input
+     *   mappings fill one target
      */
     manifest(sources) {
         const problems = [];
         const names = new Set();
-        const inputs = new Set();
+        const targets = [];
         const mappings = [];
         for (const source of sources) {
             if (names.has(source.name)) {
@@ -165,9 +209,9 @@ export const Files = {
             }
             names.add(source.name);
             for (const mapping of source.mappings) {
-                if (inputs.has(mapping.input))
-                    problems.push(variant('duplicate_input', { input: mapping.input }));
-                inputs.add(mapping.input);
+                if (targets.some((t) => sameFilesTarget(t, mapping.target)))
+                    problems.push(variant('duplicate_target', { target: mapping.target }));
+                targets.push(mapping.target);
                 mappings.push(mapping);
             }
         }