@elaraai/e3-cloud-files 1.0.91-alpha.0 → 1.0.92

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -53,9 +53,9 @@ export default Files.manifest([upload]);
53
53
 
54
54
  | Shape | Selects | The input's type | Largest file |
55
55
  |---|---|---|---|
56
- | `file(input, { key })` | One key | `Blob` | 1 GiB |
57
- | `latest(input, { prefix, include? })` | The greatest key under the prefix that matches | `Blob` | 1 GiB |
58
- | `folder(input, { prefix, include? })` | Every key under the prefix that matches, by its path under the prefix | `Dict<String, Blob>` | 64 MiB each |
56
+ | `file(input, { key })` | One key | `Blob` | 1 GiB less 64 KiB |
57
+ | `latest(input, { prefix, include? })` | The greatest key under the prefix that matches | `Blob` | 1 GiB less 64 KiB |
58
+ | `folder(input, { prefix, include? })` | Every key under the prefix that matches, by its path under the prefix | `Dict<String, Blob>` | 64 MiB less 64 KiB each |
59
59
  | `listing(input, { prefix, include? })` | The same keys' size, time and ETag, without their bytes | `Dict<String, Struct{size, modified, etag}>` | none |
60
60
 
61
61
  - **Each shape returns an ordinary `e3.input`.** It has no default, so it
@@ -65,9 +65,11 @@ export default Files.manifest([upload]);
65
65
  - **Keys and prefixes are relative to the binding.** The code never names a
66
66
  bucket. A prefix is `''` (everything the binding holds) or ends with `/`.
67
67
  - **`include` is a glob** over each file's path under the prefix: `*` matches
68
- within one folder, `**` across folders, `?` one character. `*.json` takes
69
- only the prefix's own files; `**/*.json` takes its subfolders' too, their
70
- paths naming the subfolder (`2026/10/a.json`).
68
+ within one folder, `**` across folders, `?` one character, `[abc]` or
69
+ `[a-z]` one of a set (`[!abc]` any but those), and `{csv,json}` any of its
70
+ alternatives, each a glob itself (`*.{csv,json}`). `*.json` takes only the
71
+ prefix's own files, and lists only its own folder; `**/*.json` takes its
72
+ subfolders' too, their paths naming the subfolder (`2026/10/a.json`).
71
73
  - **Mistakes are refused as they are declared** (`FilesDeclarationError`, one
72
74
  line per problem): a name e3 refuses, a key that isn't relative, a glob
73
75
  that can't match, and in `Files.manifest` two sources of one name or two
@@ -120,7 +122,7 @@ e3 workspace deploy ./repo dev --from-source src/package.ts
120
122
  e3-cloud files deploy ./repo dev src/files.ts
121
123
  e3-cloud files bind ./repo dev upload s3://partner-drop/orders-team/ --profile partner --latest 30
122
124
  e3-cloud files sync ./repo dev
123
- e3 start ./repo dev
125
+ e3 dataflow run ./repo dev
124
126
  ```
125
127
 
126
128
  - **`<repo>` is a local repository's path or a repository URL**
@@ -139,7 +141,7 @@ e3 start ./repo dev
139
141
  input took last, by ETag and size. It reads only what is new or changed,
140
142
  writes only the inputs whose files changed, and says what it did per input.
141
143
  `--input` syncs only those inputs; `--dry-run` lists what a sync would take
142
- and takes nothing. A sync doesn't run the dataflow: `e3 start` does, and the
144
+ and takes nothing. A sync doesn't run the dataflow: `e3 dataflow run` does, and the
143
145
  sync prints the command when anything changed.
144
146
  - **`files status`** shows the declarations, where each source is bound, and
145
147
  the last sync.
@@ -150,14 +152,71 @@ is written, so a sync that stops part way takes the rest next time.
150
152
  ### What a sync takes again whole
151
153
 
152
154
  A sync trusts what an input took last only while the input still holds the
153
- value the sync wrote. It takes an input's files again whole when:
155
+ value the sync wrote, and its declaration and binding are the ones it was
156
+ taken under, which it remembers with it. It takes an input's files again
157
+ whole when:
154
158
 
155
159
  - a deploy reset the input, or someone set it by hand. A package deploy resets
156
160
  inputs unless told to keep them: `e3 workspace deploy … --inputs
157
161
  keep-edited` keeps synced inputs while their type is unchanged;
158
- - the input's declaration changed (`files deploy` says which);
162
+ - the input's declaration changed (`files deploy` says which), even when a
163
+ sync under the old one was still writing it;
159
164
  - its source was bound somewhere else (`files bind` says which).
160
165
 
166
+ ### Files a sync can't read
167
+
168
+ A file that can't be read is refused, named in the report, and the sync goes
169
+ on with the rest: a file archived to a storage class that must be restored
170
+ first, one encrypted with a key the reader may not use, one that changed or
171
+ went between the listing and its read. The input keeps what it held of it,
172
+ and the next sync tries it again. Any other failure (the bucket unreachable,
173
+ throttled) stops the sync, which is safe to run again.
174
+
175
+ ### Staging and production
176
+
177
+ On a deployment, a workspace may have a staging: another workspace where
178
+ changes are tried before they go live (`e3-cloud staging`). The declarations
179
+ are part of the solution's code, so they travel as its package and apps do.
180
+ Where the files are is each workspace's own.
181
+
182
+ - **A release** (`e3-cloud release <url> <staging> <production>`) gives
183
+ production the staging's declarations, with its package and apps. Its plan
184
+ says what happens to each input: added, changed, unchanged or removed. The
185
+ release is refused while the declarations don't fit the package production
186
+ will run, or read a source production hasn't bound. Production's next sync
187
+ takes an input whose declaration changed again whole. A release syncs
188
+ nothing: sync production after it.
189
+ - **Production's bindings are its own**: a release never changes them. Bind a
190
+ new source in production before the release that brings the declarations
191
+ reading it; `files bind` takes a source no declaration reads yet.
192
+ - **A protected workspace refuses `files deploy`**: its declarations change
193
+ only by a release from its staging, or by a rollback. In an emergency an
194
+ admin can force a deploy past it (`--force`). Its history records the
195
+ force, and the next release from staging undoes it.
196
+ - **A rollback** (`e3-cloud rollback <url> <production>`) restores the
197
+ declarations of the entry it goes back to, with that entry's package and
198
+ apps.
199
+ - **A copy** of production, such as a staging started from it, gives the
200
+ target everything production has of its Files. That is its declarations,
201
+ its bindings, its last sync and what each input took. So a staging
202
+ rehearses on production's files, and its first sync takes only what
203
+ changed since production's.
204
+ - The workspace's history (`e3-cloud history`) lists every deploy of
205
+ declarations, every binding moved, and every forced change.
206
+
207
+ ```bash
208
+ # Try a change on staging
209
+ e3-cloud files deploy https://e3.example.com/repos/acme staging src/files.ts
210
+ e3-cloud files sync https://e3.example.com/repos/acme staging
211
+ e3 dataflow run https://e3.example.com/repos/acme staging
212
+
213
+ # Then production: bind any new source first, release, then sync
214
+ e3-cloud files bind https://e3.example.com/repos/acme production invoices s3://drop/acme/invoices/
215
+ e3-cloud release https://e3.example.com/repos/acme staging production
216
+ e3-cloud files sync https://e3.example.com/repos/acme production
217
+ e3 dataflow run https://e3.example.com/repos/acme production
218
+ ```
219
+
161
220
  ### Large folders
162
221
 
163
222
  A folder is stored as e3 stores any large collection, in pieces each holding
@@ -30,7 +30,7 @@ export type FilesInput<Name extends string, T extends EastType> = ReturnType<typ
30
30
  export interface FilesSelection {
31
31
  /** The folder, relative to where the workspace binds the source: `''` for all of it, else ending with `/` */
32
32
  readonly prefix: string;
33
- /** A glob over paths relative to the prefix: `*` within a segment, `**` across segments, `?` one character */
33
+ /** A glob over paths relative to the prefix: `*` within a segment, `**` across segments, `?` one character, `[a-z]` one of a set, `{csv,json}` any of its alternatives */
34
34
  readonly include?: string;
35
35
  }
36
36
  /** Thrown when declarations are not well-formed, naming every problem. */
@@ -1 +1 @@
1
- {"version":3,"file":"authoring.d.ts","sourceRoot":"","sources":["../../src/authoring.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;GAkBG;AAEH,OAAO,EAAE,MAAM,aAAa,CAAC;AAC7B,OAAO,EAAuB,KAAK,QAAQ,EAAE,MAAM,eAAe,CAAC;AAEnE,OAAO,EACL,aAAa,EAAE,eAAe,EAAE,gBAAgB,EAEhD,KAAK,aAAa,EAAE,KAAK,YAAY,EAAE,KAAK,YAAY,EACzD,MAAM,yBAAyB,CAAC;AAGjC,yFAAyF;AACzF,MAAM,MAAM,UAAU,CAAC,IAAI,SAAS,MAAM,EAAE,CAAC,SAAS,QAAQ,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;AAEvG,uGAAuG;AACvG,MAAM,WAAW,cAAc;IAC7B,6GAA6G;IAC7G,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,8GAA8G;IAC9G,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED,0EAA0E;AAC1E,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,oBAAoB;IACpB,QAAQ,CAAC,QAAQ,EAAE,SAAS,YAAY,EAAE,CAAC;gBAE/B,QAAQ,EAAE,SAAS,YAAY,EAAE;CAK9C;AA8BD;;;GAGG;AACH,qBAAa,WAAW;;IACtB,wBAAwB;IACxB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAGtB,0CAA0C;gBAC9B,IAAI,EAAE,MAAM;IAKxB,qEAAqE;IACrE,IAAI,QAAQ,IAAI,SAAS,YAAY,EAAE,CAEtC;IASD;;;;;;OAMG;IACH,IAAI,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE;QAAE,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAA;KAAE,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAM9G;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAItG;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,eAAe,CAAC;IAIxG;;;;;;;;OAQG;IACH,OAAO,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,gBAAgB,CAAC;CAG3G;AAED,uBAAuB;AACvB,eAAO,MAAM,KAAK;IAChB;;;;;;OAMG;iBACU,MAAM,GAAG,WAAW;IAIjC;;;;;;;;OAQG;sBACe,SAAS,WAAW,EAAE,GAAG,aAAa;CAmBzD,CAAC"}
1
+ {"version":3,"file":"authoring.d.ts","sourceRoot":"","sources":["../../src/authoring.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;GAkBG;AAEH,OAAO,EAAE,MAAM,aAAa,CAAC;AAC7B,OAAO,EAAuB,KAAK,QAAQ,EAAE,MAAM,eAAe,CAAC;AAEnE,OAAO,EACL,aAAa,EAAE,eAAe,EAAE,gBAAgB,EAEhD,KAAK,aAAa,EAAE,KAAK,YAAY,EAAE,KAAK,YAAY,EACzD,MAAM,yBAAyB,CAAC;AAGjC,yFAAyF;AACzF,MAAM,MAAM,UAAU,CAAC,IAAI,SAAS,MAAM,EAAE,CAAC,SAAS,QAAQ,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;AAEvG,uGAAuG;AACvG,MAAM,WAAW,cAAc;IAC7B,6GAA6G;IAC7G,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,0KAA0K;IAC1K,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAC;CAC3B;AAED,0EAA0E;AAC1E,qBAAa,qBAAsB,SAAQ,KAAK;IAC9C,oBAAoB;IACpB,QAAQ,CAAC,QAAQ,EAAE,SAAS,YAAY,EAAE,CAAC;gBAE/B,QAAQ,EAAE,SAAS,YAAY,EAAE;CAK9C;AA8BD;;;GAGG;AACH,qBAAa,WAAW;;IACtB,wBAAwB;IACxB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAGtB,0CAA0C;gBAC9B,IAAI,EAAE,MAAM;IAKxB,qEAAqE;IACrE,IAAI,QAAQ,IAAI,SAAS,YAAY,EAAE,CAEtC;IASD;;;;;;OAMG;IACH,IAAI,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE;QAAE,QAAQ,CAAC,GAAG,EAAE,MAAM,CAAA;KAAE,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAM9G;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,aAAa,CAAC;IAItG;;;;;;;OAOG;IACH,MAAM,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,eAAe,CAAC;IAIxG;;;;;;;;OAQG;IACH,OAAO,CAAC,IAAI,SAAS,MAAM,EAAE,KAAK,EAAE,IAAI,EAAE,IAAI,EAAE,cAAc,GAAG,UAAU,CAAC,IAAI,EAAE,OAAO,gBAAgB,CAAC;CAG3G;AAED,uBAAuB;AACvB,eAAO,MAAM,KAAK;IAChB;;;;;;OAMG;iBACU,MAAM,GAAG,WAAW;IAIjC;;;;;;;;OAQG;sBACe,SAAS,WAAW,EAAE,GAAG,aAAa;CAmBzD,CAAC"}
@@ -35,13 +35,24 @@ export declare function prefixProblem(prefix: string): string | null;
35
35
  export declare function includeProblem(include: string): string | null;
36
36
  /**
37
37
  * A glob's matcher over paths relative to a prefix: `*` matches within one
38
- * segment of the path, `**` across segments, `?` one character but `/`, and
39
- * every other character itself. No glob matches every path.
38
+ * folder of the path, `**` across folders, `?` one character but `/`,
39
+ * `[abc]`, `[a-z]` and `[!abc]` one character of a set but `/`, and
40
+ * `{csv,json}` any of its alternatives, each a glob itself. Every other
41
+ * character matches itself. No glob matches every path.
40
42
  *
41
43
  * @param include - The glob, or `null` for every path
42
44
  * @returns Whether a path matches
45
+ * @throws {Error} When the glob is malformed ({@link includeProblem} says how)
43
46
  */
44
47
  export declare function globMatcher(include: string | null): (path: string) => boolean;
48
+ /**
49
+ * Whether a glob matches only paths in the prefix's own folder, never in one
50
+ * under it: a listing of that folder alone holds every file it can match.
51
+ *
52
+ * @param include - The glob, or `null` for every path
53
+ * @returns Whether it names no `/` and no `**`
54
+ */
55
+ export declare function matchesTopFolderOnly(include: string | null): boolean;
45
56
  /** A binding's URL, read: its bucket, and the prefix under it, `''` or ending with `/`. */
46
57
  export interface BucketLocation {
47
58
  readonly bucket: string;
@@ -1 +1 @@
1
- {"version":3,"file":"paths.d.ts","sourceRoot":"","sources":["../../src/paths.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;GASG;AAEH;;;;;GAKG;AACH,wBAAgB,UAAU,CAAC,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAKrD;AAED;;;;;GAKG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAI3D;AAED;;;;;GAKG;AACH,wBAAgB,cAAc,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAK7D;AAED;;;;;;;GAOG;AACH,wBAAgB,WAAW,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,GAAG,CAAC,IAAI,EAAE,MAAM,KAAK,OAAO,CAwB7E;AAED,2FAA2F;AAC3F,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;CACzB;AAED;;;;;GAKG;AACH,wBAAgB,cAAc,CAAC,GAAG,EAAE,MAAM,GAAG,cAAc,GAAG;IAAE,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;CAAE,CAQzF"}
1
+ {"version":3,"file":"paths.d.ts","sourceRoot":"","sources":["../../src/paths.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;GASG;AAEH;;;;;GAKG;AACH,wBAAgB,UAAU,CAAC,GAAG,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAKrD;AAED;;;;;GAKG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAI3D;AAsFD;;;;;GAKG;AACH,wBAAgB,cAAc,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM,GAAG,IAAI,CAY7D;AAED;;;;;;;;;;GAUG;AACH,wBAAgB,WAAW,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,GAAG,CAAC,IAAI,EAAE,MAAM,KAAK,OAAO,CAI7E;AAED;;;;;;GAMG;AACH,wBAAgB,oBAAoB,CAAC,OAAO,EAAE,MAAM,GAAG,IAAI,GAAG,OAAO,CAEpE;AAED,2FAA2F;AAC3F,MAAM,WAAW,cAAc;IAC7B,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;CACzB;AAED;;;;;GAKG;AACH,wBAAgB,cAAc,CAAC,GAAG,EAAE,MAAM,GAAG,cAAc,GAAG;IAAE,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAA;CAAE,CAQzF"}
package/dist/src/paths.js CHANGED
@@ -40,6 +40,99 @@ export function prefixProblem(prefix) {
40
40
  return "doesn't end with /: a prefix names a folder";
41
41
  return null;
42
42
  }
43
+ /** Why a glob is malformed, completing "include '<include>' …". */
44
+ class GlobError extends Error {
45
+ }
46
+ /** A literal character, as a regular expression matches it. */
47
+ function literal(c) {
48
+ return c.replace(/[.+^${}()|[\]\\]/g, '\\$&');
49
+ }
50
+ /**
51
+ * A `[…]` class at `glob[at.i]`, as a regular expression: one character of a
52
+ * segment, never `/`. `[!…]` and `[^…]` match any character but those named;
53
+ * a `]` first in the class is one of its characters; `a-z` is a range.
54
+ */
55
+ function characterClass(glob, at) {
56
+ let i = at.i + 1;
57
+ let negate = false;
58
+ if (glob[i] === '!' || glob[i] === '^') {
59
+ negate = true;
60
+ i += 1;
61
+ }
62
+ let body = '';
63
+ for (let first = true;; first = false, i += 1) {
64
+ const c = glob[i];
65
+ if (c === undefined)
66
+ throw new GlobError('has a [ it never closes');
67
+ if (c === ']' && !first)
68
+ break;
69
+ if (c === '/')
70
+ throw new GlobError('has a / in a [...] class, which matches one character within a folder');
71
+ body += c === '\\' || c === ']' || c === '[' || c === '^' ? `\\${c}` : c;
72
+ }
73
+ at.i = i + 1;
74
+ return negate ? `[^/${body}]` : `[${body}]`;
75
+ }
76
+ /**
77
+ * A glob's pattern from `glob[at.i]`, as a regular expression, up to its end,
78
+ * or, inside braces, up to the `,` or `}` that ends an alternative.
79
+ */
80
+ function translate(glob, at, inBraces) {
81
+ let pattern = '';
82
+ while (at.i < glob.length) {
83
+ const c = glob[at.i];
84
+ if (inBraces && (c === ',' || c === '}'))
85
+ return pattern;
86
+ if (c === '*' && glob[at.i + 1] === '*') {
87
+ // `**/` matches zero or more whole folders; a bare `**`, anything
88
+ if (glob[at.i + 2] === '/') {
89
+ pattern += '(?:.*/)?';
90
+ at.i += 3;
91
+ }
92
+ else {
93
+ pattern += '.*';
94
+ at.i += 2;
95
+ }
96
+ }
97
+ else if (c === '*') {
98
+ pattern += '[^/]*';
99
+ at.i += 1;
100
+ }
101
+ else if (c === '?') {
102
+ pattern += '[^/]';
103
+ at.i += 1;
104
+ }
105
+ else if (c === '[') {
106
+ pattern += characterClass(glob, at);
107
+ }
108
+ else if (c === '{') {
109
+ at.i += 1;
110
+ const alternatives = [];
111
+ for (;;) {
112
+ alternatives.push(translate(glob, at, true));
113
+ const end = glob[at.i];
114
+ if (end === undefined)
115
+ throw new GlobError('has a { it never closes');
116
+ at.i += 1;
117
+ if (end === '}')
118
+ break;
119
+ }
120
+ pattern += `(?:${alternatives.join('|')})`;
121
+ }
122
+ else if (c === '}') {
123
+ throw new GlobError('has a } with no { before it');
124
+ }
125
+ else {
126
+ pattern += literal(c);
127
+ at.i += 1;
128
+ }
129
+ }
130
+ return pattern;
131
+ }
132
+ /** A glob, as the regular expression its matcher tests paths with. */
133
+ function globRegex(include) {
134
+ return new RegExp(`^${translate(include, { i: 0 }, false)}$`, 'su');
135
+ }
43
136
  /**
44
137
  * Why a glob cannot select files, or `null` when it can.
45
138
  *
@@ -53,46 +146,44 @@ export function includeProblem(include) {
53
146
  return 'begins with /: it matches paths relative to the prefix';
54
147
  if (/\*{3,}/.test(include))
55
148
  return 'has three or more * in a row';
149
+ try {
150
+ globRegex(include);
151
+ }
152
+ catch (err) {
153
+ if (err instanceof GlobError)
154
+ return err.message;
155
+ // A class whose range runs backwards (`[z-a]`) is no pattern at all
156
+ return 'has a [...] class no character can match';
157
+ }
56
158
  return null;
57
159
  }
58
160
  /**
59
161
  * A glob's matcher over paths relative to a prefix: `*` matches within one
60
- * segment of the path, `**` across segments, `?` one character but `/`, and
61
- * every other character itself. No glob matches every path.
162
+ * folder of the path, `**` across folders, `?` one character but `/`,
163
+ * `[abc]`, `[a-z]` and `[!abc]` one character of a set but `/`, and
164
+ * `{csv,json}` any of its alternatives, each a glob itself. Every other
165
+ * character matches itself. No glob matches every path.
62
166
  *
63
167
  * @param include - The glob, or `null` for every path
64
168
  * @returns Whether a path matches
169
+ * @throws {Error} When the glob is malformed ({@link includeProblem} says how)
65
170
  */
66
171
  export function globMatcher(include) {
67
172
  if (include === null)
68
173
  return () => true;
69
- let pattern = '';
70
- for (let i = 0; i < include.length; i++) {
71
- const c = include[i];
72
- if (c === '*' && include[i + 1] === '*') {
73
- // `**/` matches zero or more whole segments; a bare `**`, anything
74
- if (include[i + 2] === '/') {
75
- pattern += '(?:.*/)?';
76
- i += 2;
77
- }
78
- else {
79
- pattern += '.*';
80
- i += 1;
81
- }
82
- }
83
- else if (c === '*') {
84
- pattern += '[^/]*';
85
- }
86
- else if (c === '?') {
87
- pattern += '[^/]';
88
- }
89
- else {
90
- pattern += c.replace(/[.+^${}()|[\]\\]/g, '\\$&');
91
- }
92
- }
93
- const regex = new RegExp(`^${pattern}$`, 'su');
174
+ const regex = globRegex(include);
94
175
  return (path) => regex.test(path);
95
176
  }
177
+ /**
178
+ * Whether a glob matches only paths in the prefix's own folder, never in one
179
+ * under it: a listing of that folder alone holds every file it can match.
180
+ *
181
+ * @param include - The glob, or `null` for every path
182
+ * @returns Whether it names no `/` and no `**`
183
+ */
184
+ export function matchesTopFolderOnly(include) {
185
+ return include !== null && !include.includes('/') && !include.includes('**');
186
+ }
96
187
  /**
97
188
  * Reads a binding's URL, `s3://bucket/prefix/`.
98
189
  *
@@ -1 +1 @@
1
- {"version":3,"file":"paths.js","sourceRoot":"","sources":["../../src/paths.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;GASG;AAEH;;;;;GAKG;AACH,MAAM,UAAU,UAAU,CAAC,GAAW;IACpC,IAAI,GAAG,KAAK,EAAE;QAAE,OAAO,UAAU,CAAC;IAClC,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC;QAAE,OAAO,0EAA0E,CAAC;IAC3G,IAAI,GAAG,CAAC,QAAQ,CAAC,GAAG,CAAC;QAAE,OAAO,sDAAsD,CAAC;IACrF,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,aAAa,CAAC,MAAc;IAC1C,IAAI,MAAM,CAAC,UAAU,CAAC,GAAG,CAAC;QAAE,OAAO,6EAA6E,CAAC;IACjH,IAAI,MAAM,KAAK,EAAE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC;QAAE,OAAO,6CAA6C,CAAC;IACjG,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,cAAc,CAAC,OAAe;IAC5C,IAAI,OAAO,KAAK,EAAE;QAAE,OAAO,UAAU,CAAC;IACtC,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC;QAAE,OAAO,wDAAwD,CAAC;IAC7F,IAAI,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC;QAAE,OAAO,8BAA8B,CAAC;IAClE,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,WAAW,CAAC,OAAsB;IAChD,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,GAAG,EAAE,CAAC,IAAI,CAAC;IACxC,IAAI,OAAO,GAAG,EAAE,CAAC;IACjB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACxC,MAAM,CAAC,GAAG,OAAO,CAAC,CAAC,CAAE,CAAC;QACtB,IAAI,CAAC,KAAK,GAAG,IAAI,OAAO,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;YACxC,mEAAmE;YACnE,IAAI,OAAO,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;gBAC3B,OAAO,IAAI,UAAU,CAAC;gBACtB,CAAC,IAAI,CAAC,CAAC;YACT,CAAC;iBAAM,CAAC;gBACN,OAAO,IAAI,IAAI,CAAC;gBAChB,CAAC,IAAI,CAAC,CAAC;YACT,CAAC;QACH,CAAC;aAAM,IAAI,CAAC,KAAK,GAAG,EAAE,CAAC;YACrB,OAAO,IAAI,OAAO,CAAC;QACrB,CAAC;aAAM,IAAI,CAAC,KAAK,GAAG,EAAE,CAAC;YACrB,OAAO,IAAI,MAAM,CAAC;QACpB,CAAC;aAAM,CAAC;YACN,OAAO,IAAI,CAAC,CAAC,OAAO,CAAC,mBAAmB,EAAE,MAAM,CAAC,CAAC;QACpD,CAAC;IACH,CAAC;IACD,MAAM,KAAK,GAAG,IAAI,MAAM,CAAC,IAAI,OAAO,GAAG,EAAE,IAAI,CAAC,CAAC;IAC/C,OAAO,CAAC,IAAI,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACpC,CAAC;AAQD;;;;;GAKG;AACH,MAAM,UAAU,cAAc,CAAC,GAAW;IACxC,MAAM,KAAK,GAAG,+BAA+B,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACxD,IAAI,KAAK,KAAK,IAAI;QAAE,OAAO,EAAE,OAAO,EAAE,mCAAmC,EAAE,CAAC;IAC5E,MAAM,MAAM,GAAG,KAAK,CAAC,CAAC,CAAE,CAAC;IACzB,IAAI,CAAC,qCAAqC,CAAC,IAAI,CAAC,MAAM,CAAC;QAAE,OAAO,EAAE,OAAO,EAAE,UAAU,MAAM,gCAAgC,EAAE,CAAC;IAC9H,MAAM,MAAM,GAAG,KAAK,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IAC9B,IAAI,MAAM,KAAK,EAAE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC;QAAE,OAAO,EAAE,OAAO,EAAE,8CAA8C,EAAE,CAAC;IAC/G,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC;AAC5B,CAAC"}
1
+ {"version":3,"file":"paths.js","sourceRoot":"","sources":["../../src/paths.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;GASG;AAEH;;;;;GAKG;AACH,MAAM,UAAU,UAAU,CAAC,GAAW;IACpC,IAAI,GAAG,KAAK,EAAE;QAAE,OAAO,UAAU,CAAC;IAClC,IAAI,GAAG,CAAC,UAAU,CAAC,GAAG,CAAC;QAAE,OAAO,0EAA0E,CAAC;IAC3G,IAAI,GAAG,CAAC,QAAQ,CAAC,GAAG,CAAC;QAAE,OAAO,sDAAsD,CAAC;IACrF,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,aAAa,CAAC,MAAc;IAC1C,IAAI,MAAM,CAAC,UAAU,CAAC,GAAG,CAAC;QAAE,OAAO,6EAA6E,CAAC;IACjH,IAAI,MAAM,KAAK,EAAE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC;QAAE,OAAO,6CAA6C,CAAC;IACjG,OAAO,IAAI,CAAC;AACd,CAAC;AAED,mEAAmE;AACnE,MAAM,SAAU,SAAQ,KAAK;CAAG;AAEhC,+DAA+D;AAC/D,SAAS,OAAO,CAAC,CAAS;IACxB,OAAO,CAAC,CAAC,OAAO,CAAC,mBAAmB,EAAE,MAAM,CAAC,CAAC;AAChD,CAAC;AAED;;;;GAIG;AACH,SAAS,cAAc,CAAC,IAAY,EAAE,EAAiB;IACrD,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC;IACjB,IAAI,MAAM,GAAG,KAAK,CAAC;IACnB,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;QACvC,MAAM,GAAG,IAAI,CAAC;QACd,CAAC,IAAI,CAAC,CAAC;IACT,CAAC;IACD,IAAI,IAAI,GAAG,EAAE,CAAC;IACd,KAAK,IAAI,KAAK,GAAG,IAAI,GAAI,KAAK,GAAG,KAAK,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC;QAC/C,MAAM,CAAC,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QAClB,IAAI,CAAC,KAAK,SAAS;YAAE,MAAM,IAAI,SAAS,CAAC,yBAAyB,CAAC,CAAC;QACpE,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK;YAAE,MAAM;QAC/B,IAAI,CAAC,KAAK,GAAG;YAAE,MAAM,IAAI,SAAS,CAAC,uEAAuE,CAAC,CAAC;QAC5G,IAAI,IAAI,CAAC,KAAK,IAAI,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3E,CAAC;IACD,EAAE,CAAC,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IACb,OAAO,MAAM,CAAC,CAAC,CAAC,MAAM,IAAI,GAAG,CAAC,CAAC,CAAC,IAAI,IAAI,GAAG,CAAC;AAC9C,CAAC;AAED;;;GAGG;AACH,SAAS,SAAS,CAAC,IAAY,EAAE,EAAiB,EAAE,QAAiB;IACnE,IAAI,OAAO,GAAG,EAAE,CAAC;IACjB,OAAO,EAAE,CAAC,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC;QAC1B,MAAM,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAE,CAAC;QACtB,IAAI,QAAQ,IAAI,CAAC,CAAC,KAAK,GAAG,IAAI,CAAC,KAAK,GAAG,CAAC;YAAE,OAAO,OAAO,CAAC;QACzD,IAAI,CAAC,KAAK,GAAG,IAAI,IAAI,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;YACxC,kEAAkE;YAClE,IAAI,IAAI,CAAC,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,GAAG,EAAE,CAAC;gBAC3B,OAAO,IAAI,UAAU,CAAC;gBACtB,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;YACZ,CAAC;iBAAM,CAAC;gBACN,OAAO,IAAI,IAAI,CAAC;gBAChB,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;YACZ,CAAC;QACH,CAAC;aAAM,IAAI,CAAC,KAAK,GAAG,EAAE,CAAC;YACrB,OAAO,IAAI,OAAO,CAAC;YACnB,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;QACZ,CAAC;aAAM,IAAI,CAAC,KAAK,GAAG,EAAE,CAAC;YACrB,OAAO,IAAI,MAAM,CAAC;YAClB,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;QACZ,CAAC;aAAM,IAAI,CAAC,KAAK,GAAG,EAAE,CAAC;YACrB,OAAO,IAAI,cAAc,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QACtC,CAAC;aAAM,IAAI,CAAC,KAAK,GAAG,EAAE,CAAC;YACrB,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;YACV,MAAM,YAAY,GAAa,EAAE,CAAC;YAClC,SAAS,CAAC;gBACR,YAAY,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,EAAE,EAAE,IAAI,CAAC,CAAC,CAAC;gBAC7C,MAAM,GAAG,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;gBACvB,IAAI,GAAG,KAAK,SAAS;oBAAE,MAAM,IAAI,SAAS,CAAC,yBAAyB,CAAC,CAAC;gBACtE,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;gBACV,IAAI,GAAG,KAAK,GAAG;oBAAE,MAAM;YACzB,CAAC;YACD,OAAO,IAAI,MAAM,YAAY,CAAC,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC;QAC7C,CAAC;aAAM,IAAI,CAAC,KAAK,GAAG,EAAE,CAAC;YACrB,MAAM,IAAI,SAAS,CAAC,6BAA6B,CAAC,CAAC;QACrD,CAAC;aAAM,CAAC;YACN,OAAO,IAAI,OAAO,CAAC,CAAC,CAAC,CAAC;YACtB,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC;QACZ,CAAC;IACH,CAAC;IACD,OAAO,OAAO,CAAC;AACjB,CAAC;AAED,sEAAsE;AACtE,SAAS,SAAS,CAAC,OAAe;IAChC,OAAO,IAAI,MAAM,CAAC,IAAI,SAAS,CAAC,OAAO,EAAE,EAAE,CAAC,EAAE,CAAC,EAAE,EAAE,KAAK,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;AACtE,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,cAAc,CAAC,OAAe;IAC5C,IAAI,OAAO,KAAK,EAAE;QAAE,OAAO,UAAU,CAAC;IACtC,IAAI,OAAO,CAAC,UAAU,CAAC,GAAG,CAAC;QAAE,OAAO,wDAAwD,CAAC;IAC7F,IAAI,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC;QAAE,OAAO,8BAA8B,CAAC;IAClE,IAAI,CAAC;QACH,SAAS,CAAC,OAAO,CAAC,CAAC;IACrB,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,IAAI,GAAG,YAAY,SAAS;YAAE,OAAO,GAAG,CAAC,OAAO,CAAC;QACjD,oEAAoE;QACpE,OAAO,0CAA0C,CAAC;IACpD,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;;;;;;;;GAUG;AACH,MAAM,UAAU,WAAW,CAAC,OAAsB;IAChD,IAAI,OAAO,KAAK,IAAI;QAAE,OAAO,GAAG,EAAE,CAAC,IAAI,CAAC;IACxC,MAAM,KAAK,GAAG,SAAS,CAAC,OAAO,CAAC,CAAC;IACjC,OAAO,CAAC,IAAI,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACpC,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,oBAAoB,CAAC,OAAsB;IACzD,OAAO,OAAO,KAAK,IAAI,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;AAC/E,CAAC;AAQD;;;;;GAKG;AACH,MAAM,UAAU,cAAc,CAAC,GAAW;IACxC,MAAM,KAAK,GAAG,+BAA+B,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;IACxD,IAAI,KAAK,KAAK,IAAI;QAAE,OAAO,EAAE,OAAO,EAAE,mCAAmC,EAAE,CAAC;IAC5E,MAAM,MAAM,GAAG,KAAK,CAAC,CAAC,CAAE,CAAC;IACzB,IAAI,CAAC,qCAAqC,CAAC,IAAI,CAAC,MAAM,CAAC;QAAE,OAAO,EAAE,OAAO,EAAE,UAAU,MAAM,gCAAgC,EAAE,CAAC;IAC9H,MAAM,MAAM,GAAG,KAAK,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IAC9B,IAAI,MAAM,KAAK,EAAE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,GAAG,CAAC;QAAE,OAAO,EAAE,OAAO,EAAE,8CAA8C,EAAE,CAAC;IAC/G,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,CAAC;AAC5B,CAAC"}
@@ -8,10 +8,17 @@
8
8
  * the developer's own profile, the platform with a session scoped to the
9
9
  * binding's bucket and prefix; both read through this.
10
10
  *
11
- * A listing is the bucket's, a page of 1,000 keys at a time, in key order. An
12
- * ETag is kept without the quotes the bucket wraps it in. A file is read
13
- * whole: at most 1 GiB for a `file` or `latest` input, 64 MiB for a folder's
14
- * file.
11
+ * A listing is the bucket's, a page of 1,000 keys at a time, in key order;
12
+ * of a prefix's own folder alone (a delimiter), when the sync asks. An ETag is
13
+ * kept without the quotes the bucket wraps it in. One file's listing is a
14
+ * listing of its key alone, not a HEAD: a session that may list only under
15
+ * its binding's prefix is answered 403 for a HEAD of a key that isn't there,
16
+ * not 404. Credentials that may read the file but list nothing are asked by
17
+ * a HEAD instead, a 404 its absence. A file is read whole, at the version its listing gave (`If-Match`
18
+ * its ETag): at most 1 GiB for a `file` or `latest` input, 64 MiB for a
19
+ * folder's file. A file changed, archived, gone or closed to the reader since
20
+ * it was listed is refused (`FilesUnreadableError`), the sync going on with
21
+ * the rest; any other refusal is thrown.
15
22
  *
16
23
  * Needs `@aws-sdk/client-s3`, which a solution's own code never imports: the
17
24
  * CLI and the platform do.
@@ -19,7 +26,7 @@
19
26
  * @packageDocumentation
20
27
  */
21
28
  import { type S3Client } from '@aws-sdk/client-s3';
22
- import type { FilesReader } from '../sync/engine.js';
29
+ import { type FilesReader } from '../sync/engine.js';
23
30
  /** Where a binding's files are: a bucket, and the prefix every key is relative to. */
24
31
  export interface S3Location {
25
32
  readonly bucket: string;
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/s3/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;GAeG;AAEH,OAAO,EAAiF,KAAK,QAAQ,EAAE,MAAM,oBAAoB,CAAC;AAClI,OAAO,KAAK,EAAe,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAElE,sFAAsF;AACtF,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,+BAA+B;IAC/B,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;CACzB;AAYD;;;;;;GAMG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,QAAQ,EAAE,QAAQ,EAAE,UAAU,GAAG,WAAW,CAkCjF"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/s3/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;;;;;GAsBG;AAEH,OAAO,EAAiF,KAAK,QAAQ,EAAE,MAAM,oBAAoB,CAAC;AAClI,OAAO,EAA0C,KAAK,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAE7F,sFAAsF;AACtF,MAAM,WAAW,UAAU;IACzB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;IACxB,+BAA+B;IAC/B,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC;CACzB;AAgCD;;;;;;GAMG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,QAAQ,EAAE,QAAQ,EAAE,UAAU,GAAG,WAAW,CAgEjF"}
@@ -8,10 +8,17 @@
8
8
  * the developer's own profile, the platform with a session scoped to the
9
9
  * binding's bucket and prefix; both read through this.
10
10
  *
11
- * A listing is the bucket's, a page of 1,000 keys at a time, in key order. An
12
- * ETag is kept without the quotes the bucket wraps it in. A file is read
13
- * whole: at most 1 GiB for a `file` or `latest` input, 64 MiB for a folder's
14
- * file.
11
+ * A listing is the bucket's, a page of 1,000 keys at a time, in key order;
12
+ * of a prefix's own folder alone (a delimiter), when the sync asks. An ETag is
13
+ * kept without the quotes the bucket wraps it in. One file's listing is a
14
+ * listing of its key alone, not a HEAD: a session that may list only under
15
+ * its binding's prefix is answered 403 for a HEAD of a key that isn't there,
16
+ * not 404. Credentials that may read the file but list nothing are asked by
17
+ * a HEAD instead, a 404 its absence. A file is read whole, at the version its listing gave (`If-Match`
18
+ * its ETag): at most 1 GiB for a `file` or `latest` input, 64 MiB for a
19
+ * folder's file. A file changed, archived, gone or closed to the reader since
20
+ * it was listed is refused (`FilesUnreadableError`), the sync going on with
21
+ * the rest; any other refusal is thrown.
15
22
  *
16
23
  * Needs `@aws-sdk/client-s3`, which a solution's own code never imports: the
17
24
  * CLI and the platform do.
@@ -19,6 +26,7 @@
19
26
  * @packageDocumentation
20
27
  */
21
28
  import { GetObjectCommand, HeadObjectCommand, ListObjectsV2Command, S3ServiceException } from '@aws-sdk/client-s3';
29
+ import { FilesUnreadableError } from '../sync/engine.js';
22
30
  /** An ETag without the quotes the bucket wraps it in. */
23
31
  function bare(etag) {
24
32
  return (etag ?? '').replace(/^"|"$/g, '');
@@ -27,6 +35,25 @@ function bare(etag) {
27
35
  function missing(err) {
28
36
  return err instanceof S3ServiceException && (err.name === 'NotFound' || err.name === 'NoSuchKey' || err.$metadata.httpStatusCode === 404);
29
37
  }
38
+ /** Whether the bucket refused a request for want of access. */
39
+ function denied(err) {
40
+ return err instanceof S3ServiceException && (err.name === 'AccessDenied' || err.$metadata.httpStatusCode === 403);
41
+ }
42
+ /**
43
+ * Why the bucket refused to read one file, when the refusal is the file's
44
+ * own; `null` for any other, which the sync does not go on past.
45
+ */
46
+ function unreadable(err) {
47
+ if (!(err instanceof S3ServiceException))
48
+ return null;
49
+ switch (err.name) {
50
+ case 'PreconditionFailed': return 'changed since it was listed: the next sync takes it';
51
+ case 'InvalidObjectState': return 'is archived: its storage class must be restored before it can be read';
52
+ case 'AccessDenied': return "can't be read with the binding's access: a key policy or the bucket's refuses it";
53
+ case 'NoSuchKey': return 'went while the sync read it';
54
+ default: return err.$metadata.httpStatusCode === 404 ? 'went while the sync read it' : null;
55
+ }
56
+ }
30
57
  /**
31
58
  * The reader of one bound source in its bucket.
32
59
  *
@@ -37,10 +64,16 @@ function missing(err) {
37
64
  export function s3FilesReader(client, location) {
38
65
  const absolute = (key) => `${location.prefix}${key}`;
39
66
  return {
40
- async *list(prefix) {
67
+ async *list(prefix, options = {}) {
41
68
  let token;
42
69
  do {
43
- const page = await client.send(new ListObjectsV2Command({ Bucket: location.bucket, Prefix: absolute(prefix), ContinuationToken: token }));
70
+ const page = await client.send(new ListObjectsV2Command({
71
+ Bucket: location.bucket,
72
+ Prefix: absolute(prefix),
73
+ ContinuationToken: token,
74
+ // The prefix's own folder: its subfolders are common prefixes, never listed
75
+ ...(options.topFolderOnly === true && { Delimiter: '/' }),
76
+ }));
44
77
  for (const object of page.Contents ?? []) {
45
78
  if (object.Key === undefined)
46
79
  continue;
@@ -55,20 +88,49 @@ export function s3FilesReader(client, location) {
55
88
  } while (token !== undefined);
56
89
  },
57
90
  async stat(key) {
91
+ // The key's own listing comes first among those it begins: it is the
92
+ // shortest of them, and a listing is in key order
93
+ let page;
58
94
  try {
59
- const head = await client.send(new HeadObjectCommand({ Bucket: location.bucket, Key: absolute(key) }));
60
- return { key, size: head.ContentLength ?? 0, modified: head.LastModified ?? new Date(0), etag: bare(head.ETag) };
95
+ page = await client.send(new ListObjectsV2Command({ Bucket: location.bucket, Prefix: absolute(key), MaxKeys: 1 }));
61
96
  }
62
97
  catch (err) {
63
- if (missing(err))
64
- return null;
65
- throw err;
98
+ if (!denied(err))
99
+ throw err;
100
+ // Credentials that may read the file and list nothing: a HEAD's 404 is its absence
101
+ try {
102
+ const head = await client.send(new HeadObjectCommand({ Bucket: location.bucket, Key: absolute(key) }));
103
+ return { key, size: head.ContentLength ?? 0, modified: head.LastModified ?? new Date(0), etag: bare(head.ETag) };
104
+ }
105
+ catch (headErr) {
106
+ if (missing(headErr))
107
+ return null;
108
+ throw headErr;
109
+ }
66
110
  }
111
+ const object = page.Contents?.[0];
112
+ if (object?.Key !== absolute(key))
113
+ return null;
114
+ return { key, size: object.Size ?? 0, modified: object.LastModified ?? new Date(0), etag: bare(object.ETag) };
67
115
  },
68
- async read(key) {
69
- const object = await client.send(new GetObjectCommand({ Bucket: location.bucket, Key: absolute(key) }));
116
+ async read(file) {
117
+ let object;
118
+ try {
119
+ object = await client.send(new GetObjectCommand({ Bucket: location.bucket, Key: absolute(file.key), IfMatch: `"${file.etag}"` }));
120
+ }
121
+ catch (err) {
122
+ const reason = unreadable(err);
123
+ if (reason !== null)
124
+ throw new FilesUnreadableError(file.key, reason);
125
+ throw err;
126
+ }
70
127
  if (object.Body === undefined)
71
- throw new Error(`s3://${location.bucket}/${absolute(key)} answered no bytes`);
128
+ throw new Error(`s3://${location.bucket}/${absolute(file.key)} answered no bytes`);
129
+ if (object.ContentLength !== undefined && object.ContentLength !== file.size) {
130
+ // Never buffered: its size was never checked
131
+ object.Body.destroy?.();
132
+ throw new FilesUnreadableError(file.key, `changed since it was listed (${object.ContentLength} bytes, not the ${file.size} listed): the next sync takes it`);
133
+ }
72
134
  return object.Body.transformToByteArray();
73
135
  },
74
136
  };
@@ -1 +1 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/s3/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;GAeG;AAEH,OAAO,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,oBAAoB,EAAE,kBAAkB,EAAiB,MAAM,oBAAoB,CAAC;AAUlI,yDAAyD;AACzD,SAAS,IAAI,CAAC,IAAwB;IACpC,OAAO,CAAC,IAAI,IAAI,EAAE,CAAC,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC;AAC5C,CAAC;AAED,6DAA6D;AAC7D,SAAS,OAAO,CAAC,GAAY;IAC3B,OAAO,GAAG,YAAY,kBAAkB,IAAI,CAAC,GAAG,CAAC,IAAI,KAAK,UAAU,IAAI,GAAG,CAAC,IAAI,KAAK,WAAW,IAAI,GAAG,CAAC,SAAS,CAAC,cAAc,KAAK,GAAG,CAAC,CAAC;AAC5I,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,aAAa,CAAC,MAAgB,EAAE,QAAoB;IAClE,MAAM,QAAQ,GAAG,CAAC,GAAW,EAAE,EAAE,CAAC,GAAG,QAAQ,CAAC,MAAM,GAAG,GAAG,EAAE,CAAC;IAC7D,OAAO;QACL,KAAK,CAAC,CAAC,IAAI,CAAC,MAAc;YACxB,IAAI,KAAyB,CAAC;YAC9B,GAAG,CAAC;gBACF,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,oBAAoB,CAAC,EAAE,MAAM,EAAE,QAAQ,CAAC,MAAM,EAAE,MAAM,EAAE,QAAQ,CAAC,MAAM,CAAC,EAAE,iBAAiB,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC;gBAC1I,KAAK,MAAM,MAAM,IAAI,IAAI,CAAC,QAAQ,IAAI,EAAE,EAAE,CAAC;oBACzC,IAAI,MAAM,CAAC,GAAG,KAAK,SAAS;wBAAE,SAAS;oBACvC,MAAM;wBACJ,GAAG,EAAE,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,MAAM,CAAC;wBAC7C,IAAI,EAAE,MAAM,CAAC,IAAI,IAAI,CAAC;wBACtB,QAAQ,EAAE,MAAM,CAAC,YAAY,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC;wBAC5C,IAAI,EAAE,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC;qBACxB,CAAC;gBACJ,CAAC;gBACD,KAAK,GAAG,IAAI,CAAC,WAAW,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,qBAAqB,CAAC,CAAC,CAAC,SAAS,CAAC;YAC7E,CAAC,QAAQ,KAAK,KAAK,SAAS,EAAE;QAChC,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,GAAG;YACZ,IAAI,CAAC;gBACH,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,iBAAiB,CAAC,EAAE,MAAM,EAAE,QAAQ,CAAC,MAAM,EAAE,GAAG,EAAE,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC;gBACvG,OAAO,EAAE,GAAG,EAAE,IAAI,EAAE,IAAI,CAAC,aAAa,IAAI,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,YAAY,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YACnH,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,IAAI,OAAO,CAAC,GAAG,CAAC;oBAAE,OAAO,IAAI,CAAC;gBAC9B,MAAM,GAAG,CAAC;YACZ,CAAC;QACH,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,GAAG;YACZ,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,gBAAgB,CAAC,EAAE,MAAM,EAAE,QAAQ,CAAC,MAAM,EAAE,GAAG,EAAE,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC;YACxG,IAAI,MAAM,CAAC,IAAI,KAAK,SAAS;gBAAE,MAAM,IAAI,KAAK,CAAC,QAAQ,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC;YAC7G,OAAO,MAAM,CAAC,IAAI,CAAC,oBAAoB,EAAE,CAAC;QAC5C,CAAC;KACF,CAAC;AACJ,CAAC"}
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/s3/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;;;;;GAsBG;AAEH,OAAO,EAAE,gBAAgB,EAAE,iBAAiB,EAAE,oBAAoB,EAAE,kBAAkB,EAAiB,MAAM,oBAAoB,CAAC;AAClI,OAAO,EAAE,oBAAoB,EAAsC,MAAM,mBAAmB,CAAC;AAS7F,yDAAyD;AACzD,SAAS,IAAI,CAAC,IAAwB;IACpC,OAAO,CAAC,IAAI,IAAI,EAAE,CAAC,CAAC,OAAO,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC;AAC5C,CAAC;AAED,6DAA6D;AAC7D,SAAS,OAAO,CAAC,GAAY;IAC3B,OAAO,GAAG,YAAY,kBAAkB,IAAI,CAAC,GAAG,CAAC,IAAI,KAAK,UAAU,IAAI,GAAG,CAAC,IAAI,KAAK,WAAW,IAAI,GAAG,CAAC,SAAS,CAAC,cAAc,KAAK,GAAG,CAAC,CAAC;AAC5I,CAAC;AAED,+DAA+D;AAC/D,SAAS,MAAM,CAAC,GAAY;IAC1B,OAAO,GAAG,YAAY,kBAAkB,IAAI,CAAC,GAAG,CAAC,IAAI,KAAK,cAAc,IAAI,GAAG,CAAC,SAAS,CAAC,cAAc,KAAK,GAAG,CAAC,CAAC;AACpH,CAAC;AAED;;;GAGG;AACH,SAAS,UAAU,CAAC,GAAY;IAC9B,IAAI,CAAC,CAAC,GAAG,YAAY,kBAAkB,CAAC;QAAE,OAAO,IAAI,CAAC;IACtD,QAAQ,GAAG,CAAC,IAAI,EAAE,CAAC;QACjB,KAAK,oBAAoB,CAAC,CAAC,OAAO,qDAAqD,CAAC;QACxF,KAAK,oBAAoB,CAAC,CAAC,OAAO,uEAAuE,CAAC;QAC1G,KAAK,cAAc,CAAC,CAAC,OAAO,kFAAkF,CAAC;QAC/G,KAAK,WAAW,CAAC,CAAC,OAAO,6BAA6B,CAAC;QACvD,OAAO,CAAC,CAAC,OAAO,GAAG,CAAC,SAAS,CAAC,cAAc,KAAK,GAAG,CAAC,CAAC,CAAC,6BAA6B,CAAC,CAAC,CAAC,IAAI,CAAC;IAC9F,CAAC;AACH,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,aAAa,CAAC,MAAgB,EAAE,QAAoB;IAClE,MAAM,QAAQ,GAAG,CAAC,GAAW,EAAE,EAAE,CAAC,GAAG,QAAQ,CAAC,MAAM,GAAG,GAAG,EAAE,CAAC;IAC7D,OAAO;QACL,KAAK,CAAC,CAAC,IAAI,CAAC,MAAM,EAAE,OAAO,GAAG,EAAE;YAC9B,IAAI,KAAyB,CAAC;YAC9B,GAAG,CAAC;gBACF,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,oBAAoB,CAAC;oBACtD,MAAM,EAAE,QAAQ,CAAC,MAAM;oBACvB,MAAM,EAAE,QAAQ,CAAC,MAAM,CAAC;oBACxB,iBAAiB,EAAE,KAAK;oBACxB,4EAA4E;oBAC5E,GAAG,CAAC,OAAO,CAAC,aAAa,KAAK,IAAI,IAAI,EAAE,SAAS,EAAE,GAAG,EAAE,CAAC;iBAC1D,CAAC,CAAC,CAAC;gBACJ,KAAK,MAAM,MAAM,IAAI,IAAI,CAAC,QAAQ,IAAI,EAAE,EAAE,CAAC;oBACzC,IAAI,MAAM,CAAC,GAAG,KAAK,SAAS;wBAAE,SAAS;oBACvC,MAAM;wBACJ,GAAG,EAAE,MAAM,CAAC,GAAG,CAAC,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,MAAM,CAAC;wBAC7C,IAAI,EAAE,MAAM,CAAC,IAAI,IAAI,CAAC;wBACtB,QAAQ,EAAE,MAAM,CAAC,YAAY,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC;wBAC5C,IAAI,EAAE,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC;qBACxB,CAAC;gBACJ,CAAC;gBACD,KAAK,GAAG,IAAI,CAAC,WAAW,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,qBAAqB,CAAC,CAAC,CAAC,SAAS,CAAC;YAC7E,CAAC,QAAQ,KAAK,KAAK,SAAS,EAAE;QAChC,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,GAAG;YACZ,qEAAqE;YACrE,kDAAkD;YAClD,IAAI,IAAI,CAAC;YACT,IAAI,CAAC;gBACH,IAAI,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,oBAAoB,CAAC,EAAE,MAAM,EAAE,QAAQ,CAAC,MAAM,EAAE,MAAM,EAAE,QAAQ,CAAC,GAAG,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;YACrH,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC;oBAAE,MAAM,GAAG,CAAC;gBAC5B,mFAAmF;gBACnF,IAAI,CAAC;oBACH,MAAM,IAAI,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,iBAAiB,CAAC,EAAE,MAAM,EAAE,QAAQ,CAAC,MAAM,EAAE,GAAG,EAAE,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC;oBACvG,OAAO,EAAE,GAAG,EAAE,IAAI,EAAE,IAAI,CAAC,aAAa,IAAI,CAAC,EAAE,QAAQ,EAAE,IAAI,CAAC,YAAY,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;gBACnH,CAAC;gBAAC,OAAO,OAAO,EAAE,CAAC;oBACjB,IAAI,OAAO,CAAC,OAAO,CAAC;wBAAE,OAAO,IAAI,CAAC;oBAClC,MAAM,OAAO,CAAC;gBAChB,CAAC;YACH,CAAC;YACD,MAAM,MAAM,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,CAAC;YAClC,IAAI,MAAM,EAAE,GAAG,KAAK,QAAQ,CAAC,GAAG,CAAC;gBAAE,OAAO,IAAI,CAAC;YAC/C,OAAO,EAAE,GAAG,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,IAAI,CAAC,EAAE,QAAQ,EAAE,MAAM,CAAC,YAAY,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,IAAI,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;QAChH,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,IAAI;YACb,IAAI,MAAM,CAAC;YACX,IAAI,CAAC;gBACH,MAAM,GAAG,MAAM,MAAM,CAAC,IAAI,CAAC,IAAI,gBAAgB,CAAC,EAAE,MAAM,EAAE,QAAQ,CAAC,MAAM,EAAE,GAAG,EAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,OAAO,EAAE,IAAI,IAAI,CAAC,IAAI,GAAG,EAAE,CAAC,CAAC,CAAC;YACpI,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACb,MAAM,MAAM,GAAG,UAAU,CAAC,GAAG,CAAC,CAAC;gBAC/B,IAAI,MAAM,KAAK,IAAI;oBAAE,MAAM,IAAI,oBAAoB,CAAC,IAAI,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;gBACtE,MAAM,GAAG,CAAC;YACZ,CAAC;YACD,IAAI,MAAM,CAAC,IAAI,KAAK,SAAS;gBAAE,MAAM,IAAI,KAAK,CAAC,QAAQ,QAAQ,CAAC,MAAM,IAAI,QAAQ,CAAC,IAAI,CAAC,GAAG,CAAC,oBAAoB,CAAC,CAAC;YAClH,IAAI,MAAM,CAAC,aAAa,KAAK,SAAS,IAAI,MAAM,CAAC,aAAa,KAAK,IAAI,CAAC,IAAI,EAAE,CAAC;gBAC7E,6CAA6C;gBAC5C,MAAM,CAAC,IAAiC,CAAC,OAAO,EAAE,EAAE,CAAC;gBACtD,MAAM,IAAI,oBAAoB,CAAC,IAAI,CAAC,GAAG,EAAE,gCAAgC,MAAM,CAAC,aAAa,mBAAmB,IAAI,CAAC,IAAI,kCAAkC,CAAC,CAAC;YAC/J,CAAC;YACD,OAAO,MAAM,CAAC,IAAI,CAAC,oBAAoB,EAAE,CAAC;QAC5C,CAAC;KACF,CAAC;AACJ,CAAC"}
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/storage/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAuBH,OAAO,EAOL,KAAK,cAAc,EACpB,MAAM,kBAAkB,CAAC;AAE1B,OAAO,KAAK,EAAE,WAAW,EAAiB,MAAM,mBAAmB,CAAC;AAgDpE;;;;;;;GAOG;AACH,wBAAgB,aAAa,CAAC,OAAO,EAAE,cAAc,EAAE,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,GAAG,WAAW,CA+B5F"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../../../src/storage/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAuBH,OAAO,EAOL,KAAK,cAAc,EACpB,MAAM,kBAAkB,CAAC;AAE1B,OAAO,KAAK,EAAE,WAAW,EAAiB,MAAM,mBAAmB,CAAC;AAgDpE;;;;;;;GAOG;AACH,wBAAgB,aAAa,CAAC,OAAO,EAAE,cAAc,EAAE,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,MAAM,GAAG,WAAW,CAkC5F"}
@@ -105,7 +105,10 @@ export function storageWriter(storage, repo, ws) {
105
105
  const sources = hash === null ? [{ elements: ordered(changes.set) }] : await folderSources(storage, repo, hash, changes);
106
106
  const stored = await storeCollection(storage, repo, FilesFolderType, sources);
107
107
  // The store knows the new folder as the manifest it is, so e3's door
108
- // takes it in as the input's value, checked and versioned, reading nothing
108
+ // takes it in as the input's value, checked and versioned, reading
109
+ // nothing. A memo of the manifest to itself, until e3-core has a door
110
+ // for a value the store holds (east-workspace#1314); gc drops it once
111
+ // the manifest is collected
109
112
  await storage.refs.adoptionWrite(repo, stored, stored);
110
113
  const adopted = await datasetAdoptKnown(storage, repo, ws, inputPath(input), stored);
111
114
  if (adopted === null)
@@ -1 +1 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/storage/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;GAkBG;AAEH,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,eAAe,EAAE,OAAO,EAAE,MAAM,eAAe,CAAC;AACjF,OAAO,EACL,eAAe,EACf,iBAAiB,EACjB,eAAe,EACf,uBAAuB,EACvB,mBAAmB,GAGpB,MAAM,kBAAkB,CAAC;AAC1B,OAAO,EAAE,aAAa,EAAE,eAAe,EAAE,gBAAgB,EAAwB,MAAM,yBAAyB,CAAC;AAGjH,0CAA0C;AAC1C,SAAS,SAAS,CAAC,KAAa;IAC9B,OAAO,CAAC,OAAO,CAAC,OAAO,EAAE,QAAQ,CAAC,EAAE,OAAO,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC;AAC/D,CAAC;AAED,MAAM,WAAW,GAAG,UAAU,CAAC,UAAU,CAAC,CAAC;AAC3C,MAAM,YAAY,GAAG,eAAe,CAAC,eAAe,CAAC,CAAC;AAEtD,iCAAiC;AACjC,SAAS,OAAO,CAAC,OAAgD;IAC/D,OAAO,CAAC,GAAG,OAAO,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAwB,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,WAAW,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;AAC1G,CAAC;AAED;;;;GAIG;AACH,KAAK,UAAU,aAAa,CAAC,OAAuB,EAAE,IAAY,EAAE,IAAY,EAAE,OAAsB;IACtG,MAAM,MAAM,GAAG,MAAM,eAAe,CAAC,IAAI,CAAC,OAAO,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC;IAC/D,IAAI,MAAM,CAAC,MAAM,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,CAAC,EAAE,QAAQ,EAAE,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAC5E,MAAM,OAAO,GAAG,IAAI,GAAG,EAA6D,CAAC;IACrF,MAAM,EAAE,GAAG,KAAK,EAAE,IAAY,EAAE,EAAE;QAChC,MAAM,CAAC,GAAG,MAAM,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC;QACxC,IAAI,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;QAC5B,IAAI,MAAM,KAAK,SAAS;YAAE,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,MAAM,GAAG,EAAE,GAAG,EAAE,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,CAAC;QAC3E,OAAO,MAAM,CAAC;IAChB,CAAC,CAAC;IACF,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,OAAO,CAAC,GAAG;QAAE,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC;IAClF,KAAK,MAAM,IAAI,IAAI,OAAO,CAAC,MAAM;QAAE,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAEtE,MAAM,OAAO,GAAuB,EAAE,CAAC;IACvC,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,KAAK,MAAM,CAAC,IAAI,CAAC,GAAG,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;QAC1D,IAAI,CAAC,GAAG,IAAI;YAAE,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,CAAC;QACvE,MAAM,OAAO,GAAG,IAAI,GAAG,CAAqB,YAAY,CAAC,MAAM,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACnF,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAE,CAAC;QAC/B,KAAK,MAAM,IAAI,IAAI,MAAM,CAAC,MAAM;YAAE,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACvD,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,GAAG;YAAE,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;QACjE,OAAO,CAAC,IAAI,CAAC,EAAE,QAAQ,EAAE,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC;QAC7C,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;IACf,CAAC;IACD,IAAI,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,MAAM;QAAE,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,EAAE,EAAE,MAAM,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC;IAC7G,OAAO,OAAO,CAAC;AACjB,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,aAAa,CAAC,OAAuB,EAAE,IAAY,EAAE,EAAU;IAC7E,MAAM,OAAO,GAAG,KAAK,EAAE,KAAa,EAA0B,EAAE,CAC9D,CAAC,MAAM,uBAAuB,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;IAC5E,MAAM,OAAO,GAAG,KAAK,EAAE,KAAa,EAAmB,EAAE;QACvD,MAAM,IAAI,GAAG,MAAM,OAAO,CAAC,KAAK,CAAC,CAAC;QAClC,IAAI,IAAI,KAAK,IAAI;YAAE,MAAM,IAAI,KAAK,CAAC,SAAS,KAAK,qCAAqC,CAAC,CAAC;QACxF,OAAO,IAAI,CAAC;IACd,CAAC,CAAC;IACF,OAAO;QACL,OAAO;QACP,KAAK,CAAC,OAAO,CAAC,KAAK,EAAE,KAAK;YACxB,MAAM,mBAAmB,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,CAAC,KAAK,CAAC,EAAE,KAAK,EAAE,aAAa,CAAC,CAAC;YACrF,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC;QACxB,CAAC;QACD,KAAK,CAAC,UAAU,CAAC,KAAK,EAAE,IAA0C;YAChE,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,WAAW,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YACvE,MAAM,mBAAmB,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,CAAC,KAAK,CAAC,EAAE,KAAK,EAAE,gBAAgB,CAAC,CAAC;YACxF,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC;QACxB,CAAC;QACD,KAAK,CAAC,YAAY,CAAC,KAAK,EAAE,OAAO;YAC/B,MAAM,IAAI,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,OAAO,CAAC,KAAK,CAAC,CAAC;YACzD,MAAM,OAAO,GAAG,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,MAAM,aAAa,CAAC,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,OAAO,CAAC,CAAC;YACzH,MAAM,MAAM,GAAG,MAAM,eAAe,CAAC,OAAO,EAAE,IAAI,EAAE,eAAe,EAAE,OAAO,CAAC,CAAC;YAC9E,qEAAqE;YACrE,2EAA2E;YAC3E,MAAM,OAAO,CAAC,IAAI,CAAC,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;YACvD,MAAM,OAAO,GAAG,MAAM,iBAAiB,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC,CAAC;YACrF,IAAI,OAAO,KAAK,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,SAAS,KAAK,8BAA8B,MAAM,EAAE,CAAC,CAAC;YAC5F,OAAO,OAAO,CAAC,IAAI,CAAC;QACtB,CAAC;KACF,CAAC;AACJ,CAAC"}
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../../../src/storage/index.ts"],"names":[],"mappings":"AAAA;;;GAGG;AAEH;;;;;;;;;;;;;;;;;;GAkBG;AAEH,OAAO,EAAE,UAAU,EAAE,UAAU,EAAE,eAAe,EAAE,OAAO,EAAE,MAAM,eAAe,CAAC;AACjF,OAAO,EACL,eAAe,EACf,iBAAiB,EACjB,eAAe,EACf,uBAAuB,EACvB,mBAAmB,GAGpB,MAAM,kBAAkB,CAAC;AAC1B,OAAO,EAAE,aAAa,EAAE,eAAe,EAAE,gBAAgB,EAAwB,MAAM,yBAAyB,CAAC;AAGjH,0CAA0C;AAC1C,SAAS,SAAS,CAAC,KAAa;IAC9B,OAAO,CAAC,OAAO,CAAC,OAAO,EAAE,QAAQ,CAAC,EAAE,OAAO,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC;AAC/D,CAAC;AAED,MAAM,WAAW,GAAG,UAAU,CAAC,UAAU,CAAC,CAAC;AAC3C,MAAM,YAAY,GAAG,eAAe,CAAC,eAAe,CAAC,CAAC;AAEtD,iCAAiC;AACjC,SAAS,OAAO,CAAC,OAAgD;IAC/D,OAAO,CAAC,GAAG,OAAO,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAwB,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,WAAW,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;AAC1G,CAAC;AAED;;;;GAIG;AACH,KAAK,UAAU,aAAa,CAAC,OAAuB,EAAE,IAAY,EAAE,IAAY,EAAE,OAAsB;IACtG,MAAM,MAAM,GAAG,MAAM,eAAe,CAAC,IAAI,CAAC,OAAO,EAAE,IAAI,EAAE,IAAI,CAAC,CAAC;IAC/D,IAAI,MAAM,CAAC,MAAM,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,CAAC,EAAE,QAAQ,EAAE,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAC5E,MAAM,OAAO,GAAG,IAAI,GAAG,EAA6D,CAAC;IACrF,MAAM,EAAE,GAAG,KAAK,EAAE,IAAY,EAAE,EAAE;QAChC,MAAM,CAAC,GAAG,MAAM,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,CAAC;QACxC,IAAI,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;QAC5B,IAAI,MAAM,KAAK,SAAS;YAAE,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,MAAM,GAAG,EAAE,GAAG,EAAE,EAAE,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,CAAC;QAC3E,OAAO,MAAM,CAAC;IAChB,CAAC,CAAC;IACF,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,OAAO,CAAC,GAAG;QAAE,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC,CAAC;IAClF,KAAK,MAAM,IAAI,IAAI,OAAO,CAAC,MAAM;QAAE,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAEtE,MAAM,OAAO,GAAuB,EAAE,CAAC;IACvC,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,KAAK,MAAM,CAAC,IAAI,CAAC,GAAG,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;QAC1D,IAAI,CAAC,GAAG,IAAI;YAAE,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,CAAC;QACvE,MAAM,OAAO,GAAG,IAAI,GAAG,CAAqB,YAAY,CAAC,MAAM,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACnF,MAAM,MAAM,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAE,CAAC;QAC/B,KAAK,MAAM,IAAI,IAAI,MAAM,CAAC,MAAM;YAAE,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACvD,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,GAAG;YAAE,OAAO,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;QACjE,OAAO,CAAC,IAAI,CAAC,EAAE,QAAQ,EAAE,OAAO,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC;QAC7C,IAAI,GAAG,CAAC,GAAG,CAAC,CAAC;IACf,CAAC;IACD,IAAI,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,MAAM;QAAE,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,IAAI,EAAE,IAAI,EAAE,IAAI,EAAE,EAAE,EAAE,MAAM,CAAC,MAAM,CAAC,MAAM,EAAE,CAAC,CAAC;IAC7G,OAAO,OAAO,CAAC;AACjB,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,aAAa,CAAC,OAAuB,EAAE,IAAY,EAAE,EAAU;IAC7E,MAAM,OAAO,GAAG,KAAK,EAAE,KAAa,EAA0B,EAAE,CAC9D,CAAC,MAAM,uBAAuB,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;IAC5E,MAAM,OAAO,GAAG,KAAK,EAAE,KAAa,EAAmB,EAAE;QACvD,MAAM,IAAI,GAAG,MAAM,OAAO,CAAC,KAAK,CAAC,CAAC;QAClC,IAAI,IAAI,KAAK,IAAI;YAAE,MAAM,IAAI,KAAK,CAAC,SAAS,KAAK,qCAAqC,CAAC,CAAC;QACxF,OAAO,IAAI,CAAC;IACd,CAAC,CAAC;IACF,OAAO;QACL,OAAO;QACP,KAAK,CAAC,OAAO,CAAC,KAAK,EAAE,KAAK;YACxB,MAAM,mBAAmB,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,CAAC,KAAK,CAAC,EAAE,KAAK,EAAE,aAAa,CAAC,CAAC;YACrF,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC;QACxB,CAAC;QACD,KAAK,CAAC,UAAU,CAAC,KAAK,EAAE,IAA0C;YAChE,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,WAAW,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;YACvE,MAAM,mBAAmB,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,CAAC,KAAK,CAAC,EAAE,KAAK,EAAE,gBAAgB,CAAC,CAAC;YACxF,OAAO,OAAO,CAAC,KAAK,CAAC,CAAC;QACxB,CAAC;QACD,KAAK,CAAC,YAAY,CAAC,KAAK,EAAE,OAAO;YAC/B,MAAM,IAAI,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,MAAM,OAAO,CAAC,KAAK,CAAC,CAAC;YACzD,MAAM,OAAO,GAAG,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,MAAM,aAAa,CAAC,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,OAAO,CAAC,CAAC;YACzH,MAAM,MAAM,GAAG,MAAM,eAAe,CAAC,OAAO,EAAE,IAAI,EAAE,eAAe,EAAE,OAAO,CAAC,CAAC;YAC9E,qEAAqE;YACrE,mEAAmE;YACnE,sEAAsE;YACtE,sEAAsE;YACtE,4BAA4B;YAC5B,MAAM,OAAO,CAAC,IAAI,CAAC,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;YACvD,MAAM,OAAO,GAAG,MAAM,iBAAiB,CAAC,OAAO,EAAE,IAAI,EAAE,EAAE,EAAE,SAAS,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC,CAAC;YACrF,IAAI,OAAO,KAAK,IAAI;gBAAE,MAAM,IAAI,KAAK,CAAC,SAAS,KAAK,8BAA8B,MAAM,EAAE,CAAC,CAAC;YAC5F,OAAO,OAAO,CAAC,IAAI,CAAC;QACtB,CAAC;KACF,CAAC;AACJ,CAAC"}