@travetto/model 5.0.0-rc.10 → 5.0.0-rc.11

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -16,7 +16,7 @@ yarn add @travetto/model
16
16
  This module provides a set of contracts/interfaces to data model persistence, modification and retrieval. This module builds heavily upon the [Schema](https://github.com/travetto/travetto/tree/main/module/schema#readme "Data type registry for runtime validation, reflection and binding."), which is used for data model validation.
17
17
 
18
18
  ## Contracts
19
- The module is mainly composed of contracts. The contracts define the expected interface for various model patterns. The primary contracts are [Basic](https://github.com/travetto/travetto/tree/main/module/model/src/service/basic.ts#L9), [CRUD](https://github.com/travetto/travetto/tree/main/module/model/src/service/crud.ts#L11), [Indexed](https://github.com/travetto/travetto/tree/main/module/model/src/service/indexed.ts#L12), [Expiry](https://github.com/travetto/travetto/tree/main/module/model/src/service/expiry.ts#L11), [Streaming](https://github.com/travetto/travetto/tree/main/module/model/src/service/stream.ts#L3) and [Bulk](https://github.com/travetto/travetto/tree/main/module/model/src/service/bulk.ts#L19).
19
+ The module is mainly composed of contracts. The contracts define the expected interface for various model patterns. The primary contracts are [Basic](https://github.com/travetto/travetto/tree/main/module/model/src/service/basic.ts#L9), [CRUD](https://github.com/travetto/travetto/tree/main/module/model/src/service/crud.ts#L11), [Indexed](https://github.com/travetto/travetto/tree/main/module/model/src/service/indexed.ts#L12), [Expiry](https://github.com/travetto/travetto/tree/main/module/model/src/service/expiry.ts#L11), [Blob](https://github.com/travetto/travetto/tree/main/module/model/src/service/blob.ts#L8) and [Bulk](https://github.com/travetto/travetto/tree/main/module/model/src/service/bulk.ts#L19).
20
20
 
21
21
  ### Basic
22
22
  All [Data Modeling Support](https://github.com/travetto/travetto/tree/main/module/model#readme "Datastore abstraction for core operations.") implementations, must honor the [Basic](https://github.com/travetto/travetto/tree/main/module/model/src/service/basic.ts#L9) contract to be able to participate in the model ecosystem. This contract represents the bare minimum for a model service.
@@ -153,38 +153,44 @@ export interface ModelExpirySupport extends ModelCrudSupport {
153
153
  }
154
154
  ```
155
155
 
156
- ### Stream
157
- Some implementations also allow for the ability to read/write binary data as a [Streaming](https://github.com/travetto/travetto/tree/main/module/model/src/service/stream.ts#L3). Given that all implementations can store [Base64](https://en.wikipedia.org/wiki/Base64) encoded data, the key differentiator here, is native support for streaming data, as well as being able to store binary data of significant sizes. This pattern is currently used by [Asset](https://github.com/travetto/travetto/tree/main/module/asset#readme "Modular library for storing and retrieving binary assets") for reading and writing asset data.
156
+ ### Blob
157
+ Some implementations also allow for the ability to read/write binary data as [Blob](https://github.com/travetto/travetto/tree/main/module/model/src/service/blob.ts#L8). Given that all implementations can store [Base64](https://en.wikipedia.org/wiki/Base64) encoded data, the key differentiator here, is native support for streaming data, as well as being able to store binary data of significant sizes.
158
158
 
159
- **Code: Stream Contract**
159
+ **Code: Blob Contract**
160
160
  ```typescript
161
- export interface ModelStreamSupport {
161
+ export interface ModelBlobSupport {
162
162
 
163
163
  /**
164
- * Upsert stream to storage
165
- * @param location The location of the stream
166
- * @param input The actual stream to write
167
- * @param meta The stream metadata
164
+ * Insert blob to storage
165
+ * @param location The location of the blob
166
+ * @param input The actual blob to write
168
167
  */
169
- upsertStream(location: string, input: Readable, meta: StreamMeta): Promise<void>;
168
+ insertBlob(location: string, input: BinaryInput, meta?: BlobMeta, errorIfExisting?: boolean): Promise<void>;
170
169
 
171
170
  /**
172
- * Get stream from asset store
173
- * @param location The location of the stream
171
+ * Upsert blob to storage
172
+ * @param location The location of the blob
173
+ * @param input The actual blob to write
174
174
  */
175
- getStream(location: string, range?: StreamRange): Promise<Readable>;
175
+ upsertBlob(location: string, input: BinaryInput, meta?: BlobMeta): Promise<void>;
176
176
 
177
177
  /**
178
- * Get metadata for stream
179
- * @param location The location of the stream
178
+ * Get blob from storage
179
+ * @param location The location of the blob
180
180
  */
181
- describeStream(location: string): Promise<StreamMeta>;
181
+ getBlob(location: string, range?: ByteRange): Promise<Blob>;
182
182
 
183
183
  /**
184
- * Delete stream by location
185
- * @param location The location of the stream
184
+ * Get metadata for blob
185
+ * @param location The location of the blob
186
186
  */
187
- deleteStream(location: string): Promise<void>;
187
+ describeBlob(location: string): Promise<BlobMeta>;
188
+
189
+ /**
190
+ * Delete blob by location
191
+ * @param location The location of the blob
192
+ */
193
+ deleteBlob(location: string): Promise<void>;
188
194
  }
189
195
  ```
190
196
 
@@ -216,8 +222,8 @@ export interface ModelType {
216
222
  The `id` is the only required field for a model, as this is a hard requirement on naming and type. This may make using existing data models impossible if types other than strings are required. Additionally, the `type` field, is intended to record the base model type, but can be remapped. This is important to support polymorphism, not only in [Data Modeling Support](https://github.com/travetto/travetto/tree/main/module/model#readme "Datastore abstraction for core operations."), but also in [Schema](https://github.com/travetto/travetto/tree/main/module/schema#readme "Data type registry for runtime validation, reflection and binding.").
217
223
 
218
224
  ## Implementations
219
- |Service|Basic|CRUD|Indexed|Expiry|Stream|Bulk|
220
- |-------|-----|----|-------|------|------|----|
225
+ |Service|Basic|CRUD|Indexed|Expiry|Blob|Bulk|
226
+ |-------|-----|----|-------|------|----|----|
221
227
  |[DynamoDB Model Support](https://github.com/travetto/travetto/tree/main/module/model-dynamodb#readme "DynamoDB backing for the travetto model module.")|X|X|X|X| | |
222
228
  |[Elasticsearch Model Source](https://github.com/travetto/travetto/tree/main/module/model-elasticsearch#readme "Elasticsearch backing for the travetto model module, with real-time modeling support for Elasticsearch mappings.")|X|X|X|X| |X|
223
229
  |[Firestore Model Support](https://github.com/travetto/travetto/tree/main/module/model-firestore#readme "Firestore backing for the travetto model module.")|X|X|X| | | |
@@ -225,142 +231,76 @@ The `id` is the only required field for a model, as this is a hard requirement o
225
231
  |[Redis Model Support](https://github.com/travetto/travetto/tree/main/module/model-redis#readme "Redis backing for the travetto model module.")|X|X|X|X| ||
226
232
  |[S3 Model Support](https://github.com/travetto/travetto/tree/main/module/model-s3#readme "S3 backing for the travetto model module.")|X|X| |X|X| |
227
233
  |[SQL Model Service](https://github.com/travetto/travetto/tree/main/module/model-sql#readme "SQL backing for the travetto model module, with real-time modeling support for SQL schemas.")|X|X|X|X| |X|
228
- |[MemoryModelService](https://github.com/travetto/travetto/tree/main/module/model/src/provider/memory.ts#L54)|X|X|X|X|X|X|
229
- |[FileModelService](https://github.com/travetto/travetto/tree/main/module/model/src/provider/file.ts#L49)|X|X| |X|X|X|
234
+ |[Memory Model Support](https://github.com/travetto/travetto/tree/main/module/model-memory#readme "Memory backing for the travetto model module.")|X|X|X|X|X|X|
235
+ |[File Model Support](https://github.com/travetto/travetto/tree/main/module/model-file#readme "File system backing for the travetto model module.")|X|X| |X|X|X|
230
236
 
231
237
  ## Custom Model Service
232
- In addition to the provided contracts, the module also provides common utilities and shared test suites. The common utilities are useful for repetitive functionality, that is unable to be shared due to not relying upon inheritance (this was an intentional design decision). This allows for all the [Data Modeling Support](https://github.com/travetto/travetto/tree/main/module/model#readme "Datastore abstraction for core operations.") implementations to completely own the functionality and also to be able to provide additional/unique functionality that goes beyond the interface.
233
-
234
- **Code: Memory Service**
235
- ```typescript
236
- import { Readable } from 'node:stream';
237
- import { buffer as toBuffer } from 'node:stream/consumers';
238
- import { Class, TimeSpan, DeepPartial, castTo } from '@travetto/runtime';
239
- import { Injectable } from '@travetto/di';
240
- import { Config } from '@travetto/config';
241
- import { ModelCrudSupport } from '../service/crud';
242
- import { ModelStreamSupport, StreamMeta, StreamRange } from '../service/stream';
243
- import { ModelType, OptionalId } from '../types/model';
244
- import { ModelExpirySupport } from '../service/expiry';
245
- import { ModelRegistry } from '../registry/model';
246
- import { ModelStorageSupport } from '../service/storage';
247
- import { ModelCrudUtil } from '../internal/service/crud';
248
- import { ModelExpiryUtil } from '../internal/service/expiry';
249
- import { NotFoundError } from '../error/not-found';
250
- import { ExistsError } from '../error/exists';
251
- import { ModelIndexedSupport } from '../service/indexed';
252
- import { ModelIndexedUtil } from '../internal/service/indexed';
253
- import { ModelStorageUtil } from '../internal/service/storage';
254
- import { enforceRange, StreamModel, STREAMS } from '../internal/service/stream';
255
- import { IndexConfig } from '../registry/types';
256
- const STREAM_META = `${STREAMS}_meta`;
257
- type StoreType = Map<string, Buffer>;
258
- @Config('model.memory')
259
- export class MemoryModelConfig {
260
- autoCreate?: boolean = true;
261
- namespace?: string;
262
- cullRate?: number | TimeSpan;
263
- }
264
- function indexName<T extends ModelType>(cls: Class<T>, idx: IndexConfig<T> | string, suffix?: string): string {
265
- return [cls.Ⲑid, typeof idx === 'string' ? idx : idx.name, suffix].filter(x => !!x).join(':');
266
- }
267
- function getFirstId(data: Map<string, unknown> | Set<string>, value?: string | number): string | undefined {
268
- let id: string | undefined;
269
- if (data instanceof Set) {
270
- id = data.values().next().value;
271
- } else {
272
- id = [...data.entries()].find(([k, v]) => value === undefined || v === value)?.[0];
273
- }
274
- return id;
275
- }
276
- /**
277
- * Standard in-memory support
278
- */
279
- @Injectable()
280
- export class MemoryModelService implements ModelCrudSupport, ModelStreamSupport, ModelExpirySupport, ModelStorageSupport, ModelIndexedSupport {
281
- sorted: new Map<string, Map<string, Map<string, number>>>(),
282
- unsorted: new Map<string, Map<string, Set<string>>>()
283
- };
284
- idSource = ModelCrudUtil.uuidSource();
285
- get client(): Map<string, StoreType>;
286
- constructor(public readonly config: MemoryModelConfig) { }
287
- async postConstruct(): Promise<void>;
288
- // CRUD Support
289
- async get<T extends ModelType>(cls: Class<T>, id: string): Promise<T>;
290
- async create<T extends ModelType>(cls: Class<T>, item: OptionalId<T>): Promise<T>;
291
- async update<T extends ModelType>(cls: Class<T>, item: T): Promise<T>;
292
- async upsert<T extends ModelType>(cls: Class<T>, item: OptionalId<T>): Promise<T>;
293
- async updatePartial<T extends ModelType>(cls: Class<T>, item: Partial<T> & { id: string }, view?: string): Promise<T>;
294
- async delete<T extends ModelType>(cls: Class<T>, id: string): Promise<void>;
295
- async * list<T extends ModelType>(cls: Class<T>): AsyncIterable<T>;
296
- // Stream Support
297
- async upsertStream(location: string, input: Readable, meta: StreamMeta): Promise<void>;
298
- async getStream(location: string, range?: StreamRange): Promise<Readable>;
299
- async describeStream(location: string): Promise<StreamMeta>;
300
- async deleteStream(location: string): Promise<void>;
301
- // Expiry
302
- async deleteExpired<T extends ModelType>(cls: Class<T>): Promise<number>;
303
- // Storage Support
304
- async createStorage(): Promise<void>;
305
- async deleteStorage(): Promise<void>;
306
- async createModel<T extends ModelType>(cls: Class<T>): Promise<void>;
307
- async truncateModel<T extends ModelType>(cls: Class<T>): Promise<void>;
308
- // Indexed
309
- async getByIndex<T extends ModelType>(cls: Class<T>, idx: string, body: DeepPartial<T>): Promise<T>;
310
- async deleteByIndex<T extends ModelType>(cls: Class<T>, idx: string, body: DeepPartial<T>): Promise<void>;
311
- upsertByIndex<T extends ModelType>(cls: Class<T>, idx: string, body: OptionalId<T>): Promise<T>;
312
- async * listByIndex<T extends ModelType>(cls: Class<T>, idx: string, body?: DeepPartial<T>): AsyncIterable<T>;
313
- }
314
- ```
238
+ In addition to the provided contracts, the module also provides common utilities and shared test suites. The common utilities are useful for repetitive functionality, that is unable to be shared due to not relying upon inheritance (this was an intentional design decision). This allows for all the [Data Modeling Support](https://github.com/travetto/travetto/tree/main/module/model#readme "Datastore abstraction for core operations.") implementations to completely own the functionality and also to be able to provide additional/unique functionality that goes beyond the interface. [Memory Model Support](https://github.com/travetto/travetto/tree/main/module/model-memory#readme "Memory backing for the travetto model module.") serves as a great example of what a full featured implementation can look like.
315
239
 
316
240
  To enforce that these contracts are honored, the module provides shared test suites to allow for custom implementations to ensure they are adhering to the contract's expected behavior.
317
241
 
318
242
  **Code: Memory Service Test Configuration**
319
243
  ```typescript
320
- import { Suite } from '@travetto/test';
321
-
322
- import { MemoryModelConfig, MemoryModelService } from '../src/provider/memory';
323
- import { ModelCrudSuite } from '../support/test/crud';
324
- import { ModelExpirySuite } from '../support/test/expiry';
325
- import { ModelStreamSuite } from '../support/test/stream';
326
- import { ModelIndexedSuite } from '../support/test/indexed';
327
- import { ModelBasicSuite } from '../support/test/basic';
328
- import { ModelPolymorphismSuite } from '../support/test/polymorphism';
329
-
330
- @Suite()
331
- export class MemoryBasicSuite extends ModelBasicSuite {
332
- serviceClass = MemoryModelService;
333
- configClass = MemoryModelConfig;
334
- }
244
+ import { DependencyRegistry } from '@travetto/di';
245
+ import { AppError, castTo, Class, classConstruct } from '@travetto/runtime';
335
246
 
336
- @Suite()
337
- export class MemoryCrudSuite extends ModelCrudSuite {
338
- serviceClass = MemoryModelService;
339
- configClass = MemoryModelConfig;
340
- }
247
+ import { isBulkSupported, isCrudSupported } from '../../src/internal/service/common';
248
+ import { ModelType } from '../../src/types/model';
249
+ import { ModelSuite } from './suite';
341
250
 
342
- @Suite()
343
- export class MemoryStreamSuite extends ModelStreamSuite {
344
- serviceClass = MemoryModelService;
345
- configClass = MemoryModelConfig;
346
- }
251
+ type ServiceClass = { serviceClass: { new(): unknown } };
347
252
 
348
- @Suite()
349
- export class MemoryExpirySuite extends ModelExpirySuite {
350
- serviceClass = MemoryModelService;
351
- configClass = MemoryModelConfig;
352
- }
253
+ @ModelSuite()
254
+ export abstract class BaseModelSuite<T> {
353
255
 
354
- @Suite()
355
- export class MemoryIndexedSuite extends ModelIndexedSuite {
356
- serviceClass = MemoryModelService;
357
- configClass = MemoryModelConfig;
358
- }
256
+ static ifNot(pred: (svc: unknown) => boolean): (x: unknown) => Promise<boolean> {
257
+ return async (x: unknown) => !pred(classConstruct(castTo<ServiceClass>(x).serviceClass));
258
+ }
259
+
260
+ serviceClass: Class<T>;
261
+ configClass: Class;
262
+
263
+ async getSize<U extends ModelType>(cls: Class<U>): Promise<number> {
264
+ const svc = (await this.service);
265
+ if (isCrudSupported(svc)) {
266
+ let i = 0;
267
+ for await (const __el of svc.list(cls)) {
268
+ i += 1;
269
+ }
270
+ return i;
271
+ } else {
272
+ throw new AppError(`Size is not supported for this service: ${this.serviceClass.name}`);
273
+ }
274
+ }
275
+
276
+ async saveAll<M extends ModelType>(cls: Class<M>, items: M[]): Promise<number> {
277
+ const svc = await this.service;
278
+ if (isBulkSupported(svc)) {
279
+ const res = await svc.processBulk(cls, items.map(x => ({ insert: x })));
280
+ return res.counts.insert;
281
+ } else if (isCrudSupported(svc)) {
282
+ const out: Promise<M>[] = [];
283
+ for (const el of items) {
284
+ out.push(svc.create(cls, el));
285
+ }
286
+ await Promise.all(out);
287
+ return out.length;
288
+ } else {
289
+ throw new Error('Service does not support crud operations');
290
+ }
291
+ }
359
292
 
360
- @Suite()
361
- export class MemoryPolymorphicSuite extends ModelPolymorphismSuite {
362
- serviceClass = MemoryModelService;
363
- configClass = MemoryModelConfig;
293
+ get service(): Promise<T> {
294
+ return DependencyRegistry.getInstance(this.serviceClass);
295
+ }
296
+
297
+ async toArray<U>(src: AsyncIterable<U> | AsyncGenerator<U>): Promise<U[]> {
298
+ const out: U[] = [];
299
+ for await (const el of src) {
300
+ out.push(el);
301
+ }
302
+ return out;
303
+ }
364
304
  }
365
305
  ```
366
306
 
@@ -403,7 +343,6 @@ Options:
403
343
 
404
344
  Providers
405
345
  --------------------
406
- * Memory
407
346
  * SQL
408
347
 
409
348
  Models
package/__index__.ts CHANGED
@@ -3,15 +3,13 @@ export * from './src/registry/model';
3
3
  export * from './src/registry/types';
4
4
  export * from './src/types/model';
5
5
  export * from './src/service/basic';
6
+ export * from './src/service/blob';
6
7
  export * from './src/service/bulk';
7
8
  export * from './src/service/crud';
8
9
  export * from './src/service/indexed';
9
10
  export * from './src/service/expiry';
10
11
  export * from './src/service/storage';
11
- export * from './src/service/stream';
12
-
13
- export * from './src/provider/file';
14
- export * from './src/provider/memory';
12
+ export * from './src/util/blob';
15
13
 
16
14
  export * from './src/error/exists';
17
15
  export * from './src/error/not-found';
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@travetto/model",
3
- "version": "5.0.0-rc.10",
3
+ "version": "5.0.0-rc.11",
4
4
  "description": "Datastore abstraction for core operations.",
5
5
  "keywords": [
6
6
  "datastore",
@@ -26,14 +26,14 @@
26
26
  "directory": "module/model"
27
27
  },
28
28
  "dependencies": {
29
- "@travetto/config": "^5.0.0-rc.10",
30
- "@travetto/di": "^5.0.0-rc.9",
31
- "@travetto/registry": "^5.0.0-rc.9",
32
- "@travetto/schema": "^5.0.0-rc.10"
29
+ "@travetto/config": "^5.0.0-rc.11",
30
+ "@travetto/di": "^5.0.0-rc.10",
31
+ "@travetto/registry": "^5.0.0-rc.10",
32
+ "@travetto/schema": "^5.0.0-rc.11"
33
33
  },
34
34
  "peerDependencies": {
35
- "@travetto/cli": "^5.0.0-rc.10",
36
- "@travetto/test": "^5.0.0-rc.9"
35
+ "@travetto/cli": "^5.0.0-rc.11",
36
+ "@travetto/test": "^5.0.0-rc.10"
37
37
  },
38
38
  "peerDependenciesMeta": {
39
39
  "@travetto/cli": {
@@ -0,0 +1,5 @@
1
+ import { Class } from '@travetto/runtime';
2
+ import { ModelType } from '../../types/model';
3
+
4
+ export const ModelBlobNamespace = '__blobs';
5
+ export const MODEL_BLOB: Class<ModelType> = class { id: string; };
@@ -4,14 +4,14 @@ import { ModelCrudSupport } from '../../service/crud';
4
4
  import type { ModelExpirySupport } from '../../service/expiry';
5
5
  import { ModelIndexedSupport } from '../../service/indexed';
6
6
  import type { ModelStorageSupport } from '../../service/storage';
7
- import type { ModelStreamSupport } from '../../service/stream';
7
+ import { ModelBlobSupport } from '../../service/blob';
8
8
 
9
9
  export class ModelBasicSupportTarget { }
10
10
  export class ModelCrudSupportTarget { }
11
11
  export class ModelBulkSupportTarget { }
12
12
  export class ModelStorageSupportTarget { }
13
+ export class ModelBlobSupportTarget { }
13
14
  export class ModelExpirySupportTarget { }
14
- export class ModelStreamSupportTarget { }
15
15
  export class ModelIndexedSupportTarget { }
16
16
 
17
17
  /**
@@ -39,19 +39,19 @@ export function isExpirySupported(o: ClassInstance): o is ModelExpirySupport {
39
39
  }
40
40
 
41
41
  /**
42
- * Type guard for determining if service supports storage operation
42
+ * Type guard for determining if service supports streaming operation
43
43
  * @param o
44
44
  */
45
- export function isStorageSupported(o: ClassInstance): o is ModelStorageSupport {
46
- return !!o && 'createStorage' in o;
45
+ export function isBlobSupported(o: ClassInstance): o is ModelBlobSupport {
46
+ return !!o && 'getBlob' in o;
47
47
  }
48
48
 
49
49
  /**
50
- * Type guard for determining if service supports streaming operation
50
+ * Type guard for determining if service supports storage operation
51
51
  * @param o
52
52
  */
53
- export function isStreamSupported(o: ClassInstance): o is ModelStreamSupport {
54
- return !!o && 'getStream' in o;
53
+ export function isStorageSupported(o: ClassInstance): o is ModelStorageSupport {
54
+ return !!o && 'createStorage' in o;
55
55
  }
56
56
 
57
57
  /**
@@ -75,7 +75,6 @@ export class ModelIndexedUtil {
75
75
  return { fields, sorted };
76
76
  }
77
77
 
78
-
79
78
  /**
80
79
  * Project item via index
81
80
  * @param cls Type to get index for
@@ -0,0 +1,41 @@
1
+ import { BinaryInput, BlobMeta, ByteRange } from '@travetto/runtime';
2
+
3
+ /**
4
+ * Support for Blobs CRD. Blob update is not supported.
5
+ *
6
+ * @concrete ../internal/service/common#ModelBlobSupportTarget
7
+ */
8
+ export interface ModelBlobSupport {
9
+
10
+ /**
11
+ * Insert blob to storage
12
+ * @param location The location of the blob
13
+ * @param input The actual blob to write
14
+ */
15
+ insertBlob(location: string, input: BinaryInput, meta?: BlobMeta, errorIfExisting?: boolean): Promise<void>;
16
+
17
+ /**
18
+ * Upsert blob to storage
19
+ * @param location The location of the blob
20
+ * @param input The actual blob to write
21
+ */
22
+ upsertBlob(location: string, input: BinaryInput, meta?: BlobMeta): Promise<void>;
23
+
24
+ /**
25
+ * Get blob from storage
26
+ * @param location The location of the blob
27
+ */
28
+ getBlob(location: string, range?: ByteRange): Promise<Blob>;
29
+
30
+ /**
31
+ * Get metadata for blob
32
+ * @param location The location of the blob
33
+ */
34
+ describeBlob(location: string): Promise<BlobMeta>;
35
+
36
+ /**
37
+ * Delete blob by location
38
+ * @param location The location of the blob
39
+ */
40
+ deleteBlob(location: string): Promise<void>;
41
+ }
@@ -0,0 +1,60 @@
1
+ import { Readable } from 'node:stream';
2
+ import path from 'node:path';
3
+
4
+ import { AppError, BinaryInput, BlobMeta, ByteRange, Util } from '@travetto/runtime';
5
+
6
+ /**
7
+ * Utilities for processing assets
8
+ */
9
+ export class ModelBlobUtil {
10
+
11
+ /**
12
+ * Get a hashed location/path for a blob
13
+ */
14
+ static getHashedLocation(meta: BlobMeta, prefix = ''): string {
15
+ const hash = meta.hash ?? Util.uuid();
16
+
17
+ let parts = hash.match(/(.{1,4})/g)!.slice();
18
+ if (parts.length > 4) {
19
+ parts = [...parts.slice(0, 4), parts.slice(4).join('')];
20
+ }
21
+
22
+ const ext = path.extname(meta.filename ?? '') || '.bin';
23
+ return `${parts.join('/')}${ext}`;
24
+ }
25
+
26
+ /**
27
+ * Convert input to a blob, containing all data in memory
28
+ */
29
+ static async getInput(src: BinaryInput, metadata: BlobMeta = {}): Promise<[Readable, BlobMeta]> {
30
+ let input: Readable;
31
+ if (src instanceof Blob) {
32
+ metadata = { ...src.meta, ...metadata };
33
+ metadata.size ??= src.size;
34
+ input = Readable.fromWeb(src.stream());
35
+ } else if (typeof src === 'object' && 'pipeThrough' in src) {
36
+ input = Readable.fromWeb(src);
37
+ } else if (typeof src === 'object' && 'pipe' in src) {
38
+ input = src;
39
+ } else {
40
+ metadata.size = src.length;
41
+ input = Readable.from(src);
42
+ }
43
+
44
+ return [input, metadata ?? {}];
45
+ }
46
+
47
+ /**
48
+ * Enforce byte range for stream stream/file of a certain size
49
+ */
50
+ static enforceRange({ start, end }: ByteRange, size: number): Required<ByteRange> {
51
+ // End is inclusive
52
+ end = Math.min(end ?? (size - 1), size - 1);
53
+
54
+ if (Number.isNaN(start) || Number.isNaN(end) || !Number.isFinite(start) || start >= size || start < 0 || start > end) {
55
+ throw new AppError('Invalid position, out of range', 'data');
56
+ }
57
+
58
+ return { start, end };
59
+ }
60
+ }
@@ -8,14 +8,14 @@ export const Links = {
8
8
  Expiry: d.codeLink('Expiry', '@travetto/model/src/service/expiry.ts', /export interface/),
9
9
  Indexed: d.codeLink('Indexed', '@travetto/model/src/service/indexed.ts', /export interface/),
10
10
  Bulk: d.codeLink('Bulk', '@travetto/model/src/service/bulk.ts', /export interface/),
11
- Stream: d.codeLink('Streaming', '@travetto/model/src/service/stream.ts', /export interface/),
11
+ Blob: d.codeLink('Blob', '@travetto/model/src/service/blob.ts', /export interface/),
12
12
  };
13
13
 
14
14
  export const ModelTypes = (fn: | Function): DocJSXElement[] => {
15
15
  const { content } = DocFileUtil.readSource(fn);
16
16
  const found: DocJSXElementByFn<'CodeLink'>[] = [];
17
17
  const seen = new Set();
18
- for (const [, key] of content.matchAll(/Model(Crud|Expiry|Indexed|Bulk|Stream)Support/g)) {
18
+ for (const [, key] of content.matchAll(/Model(Crud|Expiry|Indexed|Bulk|Blob)Support/g)) {
19
19
  if (!seen.has(key)) {
20
20
  seen.add(key);
21
21
  // eslint-disable-next-line @typescript-eslint/consistent-type-assertions
@@ -0,0 +1 @@
1
+ abcdefghijklmnopqrstuvwxyz
File without changes
File without changes
Binary file
Binary file
Binary file
Binary file