electrodb 1.8.2 → 1.9.0

package/README.md

@@ -1071,7 +1071,7 @@ When using ElectroDB, indexes are referenced by their `AccessPatternName`. This
 
 All DynamoDB table start with at least a PartitionKey with an optional SortKey, this can be referred to as the _"Table Index"_. The `indexes` object requires at least the definition of this _Table Index_ **Partition Key** and (if applicable) **Sort Key**.
 
-In your model, the _Table Index_ this is expressed as an _Access Pattern_ *without* an `index` property. For Secondary Indexes, use the `index` property to define the name of the index as defined on your DynamoDB table.
+In your model, the _Table Index_ this is expressed as an _Access Pattern_ *without* an `index` property. For Secondary Indexes (both GSIs and LSIs), use the `index` property to define the name of the index as defined on your DynamoDB table.
 
 Within these _AccessPatterns_, you define the PartitionKey and (optionally) SortKeys that are present on your DynamoDB table and map the key's name on the table with the `field` property.
 
@@ -2739,7 +2739,7 @@ const MallStore = new Entity(schema, {table: "StoreDirectory"});
 #### Partition Key Composite Attributes
 All queries require (*at minimum*) the **Composite Attributes** included in its defined **Partition Key**. **Composite Attributes** you define on the **Sort Key** can be partially supplied, but must be supplied in the order they are defined.
 
-> *Important: Composite Attributes must be supplied in the order they are composed when invoking the **Access Pattern***. This is because composite attributes are used to form a concatenated key string, and if attributes supplied out of order, it is not possible to fill the gaps in that concatenation.
+> *IMPORTANT: Composite Attributes must be supplied in the order they are composed when invoking the **Access Pattern***. This is because composite attributes are used to form a concatenated key string, and if attributes supplied out of order, it is not possible to fill the gaps in that concatenation.
 
 ```javascript
 const MallStore = new Entity({
@@ -2908,6 +2908,8 @@ The two-dimensional array returned by batch get most easily used when deconstruc
 
 The `results` array are records that were returned DynamoDB as `Responses` on the BatchGet query. They will appear in the same format as other ElectroDB queries.
 
+> _NOTE: By default ElectroDB will return items without concern for order. If the order returned by ElectroDB must match the order provided, the [query option](#query-options) `preserveBatchOrder` can be used. When enabled, ElectroDB will ensure the order returned by a batchGet will be the same as the order provided. When enabled, if a record is returned from DynamoDB as "unprocessed" ([read more here](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchGetItem.html)), ElectroDB will return a null value at that index._
+
 Elements of the `unprocessed` array are unlike results received from a query. Instead of containing all the attributes of a record, an unprocessed record only includes the composite attributes defined in the Table Index. This is in keeping with DynamoDB's practice of returning only Keys in the case of unprocessed records. For convenience, ElectroDB will return these keys as composite attributes, but you can pass the [query option](#query-options) `{unprocessed:"raw"}` override this behavior and return the Keys as they came from DynamoDB.
 
 ### Delete Method
@@ -3245,7 +3247,7 @@ entity.update({ attr1: "value1", attr2: "value2" })
 }
 ```
 
-> Note: Included in the update are all attributes from the table's primary index. These values are automatically included on all updates in the event an update results in an insert. 
+> _NOTE: Included in the update are all attributes from the table's primary index. These values are automatically included on all updates in the event an update results in an insert.__
 
 #### Update Method: Set
 
@@ -4297,25 +4299,27 @@ By default, **ElectroDB** enables you to work with records as the names and prop
   pages?: number;
   logger?: (event) => void;
   listeners Array<(event) => void>;
+  preserveBatchOrder?: boolean;
 };
 ```
 
-Option          | Default              | Description
---------------- | :------------------: | -----------   
-params          | `{}`                 | Properties added to this object will be merged onto the params sent to the document client. Any conflicts with **ElectroDB** will favor the params specified here.
-table           | _(from constructor)_ | Use a different table than the one defined in the [Service Options](#service-options)
-raw             | `false`              | Returns query results as they were returned by the docClient.
-includeKeys     | `false`              | By default, **ElectroDB** does not return partition, sort, or global keys in its response.
-pager           | `"named"`            | Used in with pagination (`.pages()`) calls to override ElectroDBs default behaviour to break apart `LastEvaluatedKeys` records into composite attributes. See more detail about this in the sections for [Pager Query Options](#pager-query-options).
-originalErr     | `false`              | By default, **ElectroDB** alters the stacktrace of any exceptions thrown by the DynamoDB client to give better visibility to the developer. Set this value equal to `true` to turn off this functionality and return the error unchanged.
-concurrent      | `1`                  | When performing batch operations, how many requests (1 batch operation == 1 request) to DynamoDB should ElectroDB make at one time. Be mindful of your DynamoDB throughput configurations
-unprocessed     | `"item"`             | Used in batch processing to override ElectroDBs default behaviour to break apart DynamoDBs `Unprocessed` records into composite attributes. See more detail about this in the sections for [BatchGet](#batch-get), [BatchDelete](#batch-write-delete-records), and [BatchPut](#batch-write-put-records).
-response        | `"default"`          | Used as a convenience for applying the DynamoDB parameter `ReturnValues`. The options here are the same as the parameter values for the DocumentClient except lowercase. The `"none"` option will cause the method to return null and will bypass ElectroDB's response formatting -- useful if formatting performance is a concern.
-ignoreOwnership | `false`              | By default, **ElectroDB** interrogates items returned from a query for the presence of matching entity "identifiers". This helps to ensure other entities, or other versions of an entity, are filtered from your results. If you are using ElectroDB with an existing table/dataset you can turn off this feature by setting this property to `true`.
-limit           | _none_               | A target for the number of items to return from DynamoDB. If this option is passed, Queries on entities and through collections will paginate DynamoDB until this limit is reached or all items for that query have been returned.
-pages           | ∞                    | How many DynamoDB pages should a query iterate through before stopping. By default ElectroDB paginate through all results for your query.
-listeners       | `[]`                 | An array of callbacks that are invoked when [internal ElectroDB events](#events) occur.
-logger          | _none_               | A convenience option for a single event listener that semantically can be used for logging.
+Option             | Default              | Description
+------------------ | :------------------: | -----------
+params             | `{}`                 | Properties added to this object will be merged onto the params sent to the document client. Any conflicts with **ElectroDB** will favor the params specified here.
+table              | _(from constructor)_ | Use a different table than the one defined in the [Service Options](#service-options)
+raw                | `false`              | Returns query results as they were returned by the docClient.
+includeKeys        | `false`              | By default, **ElectroDB** does not return partition, sort, or global keys in its response.
+pager              | `"named"`            | Used in with pagination (`.pages()`) calls to override ElectroDBs default behaviour to break apart `LastEvaluatedKeys` records into composite attributes. See more detail about this in the sections for [Pager Query Options](#pager-query-options).
+originalErr        | `false`              | By default, **ElectroDB** alters the stacktrace of any exceptions thrown by the DynamoDB client to give better visibility to the developer. Set this value equal to `true` to turn off this functionality and return the error unchanged.
+concurrent         | `1`                  | When performing batch operations, how many requests (1 batch operation == 1 request) to DynamoDB should ElectroDB make at one time. Be mindful of your DynamoDB throughput configurations
+unprocessed        | `"item"`             | Used in batch processing to override ElectroDBs default behaviour to break apart DynamoDBs `Unprocessed` records into composite attributes. See more detail about this in the sections for [BatchGet](#batch-get), [BatchDelete](#batch-write-delete-records), and [BatchPut](#batch-write-put-records).
+response           | `"default"`          | Used as a convenience for applying the DynamoDB parameter `ReturnValues`. The options here are the same as the parameter values for the DocumentClient except lowercase. The `"none"` option will cause the method to return null and will bypass ElectroDB's response formatting -- useful if formatting performance is a concern.
+ignoreOwnership    | `false`              | By default, **ElectroDB** interrogates items returned from a query for the presence of matching entity "identifiers". This helps to ensure other entities, or other versions of an entity, are filtered from your results. If you are using ElectroDB with an existing table/dataset you can turn off this feature by setting this property to `true`.
+limit              | _none_               | A target for the number of items to return from DynamoDB. If this option is passed, Queries on entities and through collections will paginate DynamoDB until this limit is reached or all items for that query have been returned.
+pages              | ∞                    | How many DynamoDB pages should a query iterate through before stopping. By default ElectroDB paginate through all results for your query.
+listeners          | `[]`                 | An array of callbacks that are invoked when [internal ElectroDB events](#events) occur.
+logger             | _none_               | A convenience option for a single event listener that semantically can be used for logging.
+preserveBatchOrder | `false`              | When used with a [batchGet](#batch-get) operation, ElectroDB will ensure the order returned by a batchGet will be the same as the order provided. When enabled, if a record is returned from DynamoDB as "unprocessed" ([read more here](https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchGetItem.html)), ElectroDB will return a null value at that index.
 
 # AWS DynamoDB Client
 ElectroDB supports both the [v2](https://www.npmjs.com/package/aws-sdk) and [v3](https://www.npmjs.com/package/@aws-sdk/client-dynamodb) aws clients. The client can be supplied creating a new Entity or Service, or added to a Entity/Service instance via the `setClient()` method.

package/index.d.ts

@@ -1023,6 +1023,7 @@ interface PaginationOptions extends QueryOptions {
 interface BulkOptions extends QueryOptions {
     unprocessed?: "raw" | "item";
     concurrency?: number;
+    preserveBatchOrder?: boolean;
 }
 
 type OptionalDefaultEntityIdentifiers = {
@@ -1032,6 +1033,15 @@ type OptionalDefaultEntityIdentifiers = {
 
 type GoRecord<ResponseType, Options = QueryOptions> = <T = ResponseType>(options?: Options) => Promise<T>;
 
+type BatchGoRecord<ResponseType, AlternateResponseType> = <O extends BulkOptions>(options?: O) =>
+    O extends infer Options
+        ? 'preserveBatchOrder' extends keyof Options
+        ? Options['preserveBatchOrder'] extends true
+            ? Promise<AlternateResponseType>
+            : Promise<ResponseType>
+        : Promise<ResponseType>
+        : never
+
 type PageRecord<ResponseType, CompositeAttributes> = (page?: (CompositeAttributes & OptionalDefaultEntityIdentifiers) | null, options?: PaginationOptions) => Promise<[
     (CompositeAttributes & OptionalDefaultEntityIdentifiers) | null,
     ResponseType
@@ -1070,8 +1080,8 @@ type DeleteRecordOperationOptions<A extends string, F extends A, C extends strin
     where: WhereClause<A,F,C,S,Item<A,F,C,S,S["attributes"]>,DeleteRecordOperationOptions<A,F,C,S,ResponseType>>;
 };
 
-type BulkRecordOperationOptions<A extends string, F extends A, C extends string, S extends Schema<A,F,C>, ResponseType> = {
-    go: GoRecord<ResponseType, BulkOptions>;
+type BulkRecordOperationOptions<A extends string, F extends A, C extends string, S extends Schema<A,F,C>, ResponseType, AlternateResponseType> = {
+    go: BatchGoRecord<ResponseType, AlternateResponseType>;
     params: ParamRecord<BulkOptions>;
 };
 
@@ -1221,9 +1231,9 @@ export class Entity<A extends string, F extends A, C extends string, S extends S
     readonly schema: S;
     constructor(schema: S, config?: EntityConfiguration);
     get(key: AllTableIndexCompositeAttributes<A,F,C,S>): SingleRecordOperationOptions<A,F,C,S, ResponseItem<A,F,C,S> | null>;
-    get(key: AllTableIndexCompositeAttributes<A,F,C,S>[]): BulkRecordOperationOptions<A,F,C,S, [Array<ResponseItem<A,F,C,S>>, Array<AllTableIndexCompositeAttributes<A,F,C,S>>]>;
+    get(key: AllTableIndexCompositeAttributes<A,F,C,S>[]): BulkRecordOperationOptions<A,F,C,S, [Array<Flatten<ResponseItem<A,F,C,S>>>, Array<Flatten<AllTableIndexCompositeAttributes<A,F,C,S>>>], [Array<Flatten<ResponseItem<A,F,C,S>> | null>, Array<Flatten<AllTableIndexCompositeAttributes<A,F,C,S>>>]>;
     delete(key: AllTableIndexCompositeAttributes<A,F,C,S>): DeleteRecordOperationOptions<A,F,C,S, ResponseItem<A,F,C,S>>;
-    delete(key: AllTableIndexCompositeAttributes<A,F,C,S>[]): BulkRecordOperationOptions<A,F,C,S, AllTableIndexCompositeAttributes<A,F,C,S>[]>;
+    delete(key: AllTableIndexCompositeAttributes<A,F,C,S>[]): BulkRecordOperationOptions<A,F,C,S, AllTableIndexCompositeAttributes<A,F,C,S>[], AllTableIndexCompositeAttributes<A,F,C,S>[]>;
     remove(key: AllTableIndexCompositeAttributes<A,F,C,S>): DeleteRecordOperationOptions<A,F,C,S, ResponseItem<A,F,C,S>>;
     update(key: AllTableIndexCompositeAttributes<A,F,C,S>): {
         set: SetRecord<A,F,C,S, SetItem<A,F,C,S>, TableIndexCompositeAttributes<A,F,C,S>, ResponseItem<A,F,C,S>>;
@@ -1244,7 +1254,7 @@ export class Entity<A extends string, F extends A, C extends string, S extends S
         data: DataUpdateMethodRecord<A,F,C,S, Item<A,F,C,S,S["attributes"]>, TableIndexCompositeAttributes<A,F,C,S>, ResponseItem<A,F,C,S>>;
     };
     put(record: PutItem<A,F,C,S>): PutRecordOperationOptions<A,F,C,S, ResponseItem<A,F,C,S>>;
-    put(record: PutItem<A,F,C,S>[]): BulkRecordOperationOptions<A,F,C,S, AllTableIndexCompositeAttributes<A,F,C,S>[]>;
+    put(record: PutItem<A,F,C,S>[]): BulkRecordOperationOptions<A,F,C,S, AllTableIndexCompositeAttributes<A,F,C,S>[], AllTableIndexCompositeAttributes<A,F,C,S>[]>;
     create(record: PutItem<A,F,C,S>): PutRecordOperationOptions<A,F,C,S, ResponseItem<A,F,C,S>>;
     find(record: Partial<Item<A,F,C,S,S["attributes"]>>): RecordsActionOptions<A,F,C,S, ResponseItem<A,F,C,S>[], AllTableIndexCompositeAttributes<A,F,C,S>>;
     match(record: Partial<Item<A,F,C,S,S["attributes"]>>): RecordsActionOptions<A,F,C,S, ResponseItem<A,F,C,S>[], AllTableIndexCompositeAttributes<A,F,C,S>>;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "electrodb",
-  "version": "1.8.2",
+  "version": "1.9.0",
   "description": "A library to more easily create and interact with multiple entities and heretical relationships in dynamodb",
   "main": "index.js",
   "scripts": {

package/src/entity.js

@@ -277,6 +277,7 @@ class Entity {
 				results,
 			}, config.listeners);
 		}
+
 		return this.client[method](params).promise()
 			.then((results) => {
 				notifyQuery();
@@ -317,14 +318,36 @@ class Entity {
 		return results;
 	}
 
+	_createNewBatchGetOrderMaintainer(config = {}) {
+		const pkName = this.model.translations.keys[TableIndex].pk;
+		const skName = this.model.translations.keys[TableIndex].sk;
+		const enabled = !!config.preserveBatchOrder;
+		const table = this.config.table;
+		const keyFormatter = ((record = {}) => {
+			const pk = record[pkName];
+			const sk = record[skName];
+			return `${pk}${sk}`;
+		});
+
+		return new u.BatchGetOrderMaintainer({
+			table,
+			enabled,
+			keyFormatter,
+		});
+	}
+
 	async executeBulkGet(parameters, config) {
 		if (!Array.isArray(parameters)) {
 			parameters = [parameters];
 		}
-		let concurrent = this._normalizeConcurrencyValue(config.concurrent)
-		let concurrentOperations = u.batchItems(parameters, concurrent);
 
-		let resultsAll = [];
+		const orderMaintainer = this._createNewBatchGetOrderMaintainer(config);
+		orderMaintainer.defineOrder(parameters);
+		let concurrent = this._normalizeConcurrencyValue(config.concurrent);
+		let concurrentOperations = u.batchItems(parameters, concurrent);
+		let resultsAll = config.preserveBatchOrder
+			? new Array(orderMaintainer.getSize()).fill(null)
+			: [];
 		let unprocessedAll = [];
 		for (let operation of concurrentOperations) {
 			await Promise.all(operation.map(async params => {
@@ -333,13 +356,13 @@ class Entity {
 					resultsAll.push(await config.parse(config, response));
 					return;
 				}
-				let [results, unprocessed] = this.formatBulkGetResponse(response, config);
-				for (let r of results) {
-					resultsAll.push(r);
-				}
-				for (let u of unprocessed) {
-					unprocessedAll.push(u);
-				}
+				this.applyBulkGetResponseFormatting({
+					orderMaintainer,
+					resultsAll,
+					unprocessedAll,
+					response,
+					config
+				});
 			}));
 		}
 		return [resultsAll, unprocessedAll];
@@ -467,20 +490,26 @@ class Entity {
 		}
 	}
 
-	formatBulkGetResponse(response = {}, config = {}) {
-		let unprocessed = [];
-		let results = [];
+	applyBulkGetResponseFormatting({
+		resultsAll,
+		unprocessedAll,
+		orderMaintainer,
+		response = {},
+		config = {},
+	}) {
 		const table = config.table || this._getTableName();
 		const index = TableIndex;
+
 		if (!response.UnprocessedKeys || !response.Responses) {
 			throw new Error("Unknown response format");
 		}
+
 		if (response.UnprocessedKeys[table] && response.UnprocessedKeys[table].Keys && Array.isArray(response.UnprocessedKeys[table].Keys)) {
 			for (let value of response.UnprocessedKeys[table].Keys) {
 				if (config && config.unprocessed === UnprocessedTypes.raw) {
-					unprocessed.push(value);
+					unprocessedAll.push(value);
 				} else {
-					unprocessed.push(
+					unprocessedAll.push(
 						this._formatKeysToItem(index, value)
 					);
 				}
@@ -488,10 +517,18 @@ class Entity {
 		}
 
 		if (response.Responses[table] && Array.isArray(response.Responses[table])) {
-			results = this.formatResponse({Items: response.Responses[table]}, index, config);
+			const responses = response.Responses[table];
+			for (let i = 0; i < responses.length; i++) {
+				const item = responses[i];
+				const slot = orderMaintainer.getOrder(item);
+				const formatted = this.formatResponse({Item: item}, index, config);
+				if (slot !== -1) {
+					resultsAll[slot] = formatted;
+				} else {
+					resultsAll.push(formatted);
+				}
+			}
 		}
-
-		return [results, unprocessed];
 	}
 
 	formatResponse(response, index, config = {}) {
@@ -782,6 +819,7 @@ class Entity {
 			_isCollectionQuery: false,
 			pages: undefined,
 			listeners: [],
+			preserveBatchOrder: false,
 		};
 
 		config = options.reduce((config, option) => {
@@ -794,6 +832,10 @@ class Entity {
 				config.params.ReturnValues = FormatToReturnValues[format];
 			}
 
+			if (option.preserveBatchOrder === true) {
+				config.preserveBatchOrder = true;
+			}
+
 			if (option.pages !== undefined) {
 				config.pages = option.pages;
 			}
@@ -2257,22 +2299,6 @@ class Entity {
 				facets.fields.push(sk.field);
 			}
 
-			if (seenIndexFields[pk.field] !== undefined) {
-				throw new e.ElectroError(e.ErrorCodes.DuplicateIndexFields, `Partition Key (pk) on Access Pattern '${accessPattern}' references the field '${pk.field}' which is already referenced by the Access Pattern '${seenIndexFields[pk.field]}'. Fields used for indexes need to be unique to avoid conflicts.`);
-			} else {
-				seenIndexFields[pk.field] = accessPattern;
-			}
-
-			if (sk.field) {
-				if (sk.field === pk.field) {
-					throw new e.ElectroError(e.ErrorCodes.DuplicateIndexFields, `The Access Pattern '${accessPattern}' references the field '${sk.field}' as the field name for both the PK and SK. Fields used for indexes need to be unique to avoid conflicts.`);
-				} else if (seenIndexFields[sk.field] !== undefined) {
-					throw new e.ElectroError(e.ErrorCodes.DuplicateIndexFields, `Sort Key (sk) on Access Pattern '${accessPattern}' references the field '${sk.field}' which is already referenced by the Access Pattern '${seenIndexFields[sk.field]}'. Fields used for indexes need to be unique to avoid conflicts.`);
-				}else {
-					seenIndexFields[sk.field] = accessPattern;
-				}
-			}
-
 			if (Array.isArray(sk.facets)) {
 				let duplicates = pk.facets.filter(facet => sk.facets.includes(facet));
 				if (duplicates.length !== 0) {
@@ -2362,6 +2388,37 @@ class Entity {
 				facets.byField[sk.field][indexName] = sk;
 			}
 
+			if (seenIndexFields[pk.field] !== undefined) {
+				const definition = Object.values(facets.byField[pk.field]).find(definition => definition.index !== indexName)
+				const definitionsMatch = validations.stringArrayMatch(pk.facets, definition.facets);
+				if (!definitionsMatch) {
+					throw new e.ElectroError(e.ErrorCodes.InconsistentIndexDefinition, `Partition Key (pk) on Access Pattern '${u.formatIndexNameForDisplay(accessPattern)}' is defined with the composite attribute(s) ${u.commaSeparatedString(pk.facets)}, but the accessPattern '${u.formatIndexNameForDisplay(definition.index)}' defines this field with the composite attributes ${u.commaSeparatedString(definition.facets)}'. Key fields must have the same composite attribute definitions across all indexes they are involved with`);
+				}
+				seenIndexFields[pk.field].push({accessPattern, type: 'pk'});
+			} else {
+				seenIndexFields[pk.field] = [];
+				seenIndexFields[pk.field].push({accessPattern, type: 'pk'});
+			}
+
+			if (sk.field) {
+				if (sk.field === pk.field) {
+					throw new e.ElectroError(e.ErrorCodes.DuplicateIndexFields, `The Access Pattern '${u.formatIndexNameForDisplay(accessPattern)}' references the field '${sk.field}' as the field name for both the PK and SK. Fields used for indexes need to be unique to avoid conflicts.`);
+				} else if (seenIndexFields[sk.field] !== undefined) {
+					const isAlsoDefinedAsPK = seenIndexFields[sk.field].find(field => field.type === "pk");
+					if (isAlsoDefinedAsPK) {
+						throw new e.ElectroError(e.ErrorCodes.InconsistentIndexDefinition, `The Sort Key (sk) on Access Pattern '${u.formatIndexNameForDisplay(accessPattern)}' references the field '${pk.field}' which is already referenced by the Access Pattern(s) '${u.formatIndexNameForDisplay(isAlsoDefinedAsPK.accessPattern)}' as a Partition Key. Fields mapped to Partition Keys cannot be also mapped to Sort Keys.`);
+					}
+					const definition = Object.values(facets.byField[sk.field]).find(definition => definition.index !== indexName)
+					const definitionsMatch = validations.stringArrayMatch(sk.facets, definition.facets);
+					if (!definitionsMatch) {
+						throw new e.ElectroError(e.ErrorCodes.DuplicateIndexFields, `Sort Key (sk) on Access Pattern '${u.formatIndexNameForDisplay(accessPattern)}' is defined with the composite attribute(s) ${u.commaSeparatedString(sk.facets)}, but the accessPattern '${u.formatIndexNameForDisplay(definition.index)}' defines this field with the composite attributes ${u.commaSeparatedString(definition.facets)}'. Key fields must have the same composite attribute definitions across all indexes they are involved with`);
+					}
+					seenIndexFields[sk.field].push({accessPattern, type: 'sk'});
+				} else {
+					seenIndexFields[sk.field] = [];
+					seenIndexFields[sk.field].push({accessPattern, type: 'sk'});
+				}
+			}
 
 			attributes.forEach(({index, type, name}, j) => {
 				let next = attributes[j + 1] !== undefined ? attributes[j + 1].name : "";

package/src/errors.js

@@ -139,6 +139,12 @@ const ErrorCodes = {
     name: "InvalidClientProvided",
     sym: ErrorCode,
   },
+  InconsistentIndexDefinition: {
+    code: 1022,
+    section: "inconsistent-index-definition",
+    name: "InvalidClientProvided",
+    sym: ErrorCode,
+  },
   MissingAttribute: {
     code: 2001,
     section: "missing-attribute",

package/src/util.js

@@ -75,8 +75,8 @@ function batchItems(arr = [], size) {
   return batched;
 }
 
-function commaSeparatedString(array = []) {
-  return array.map(value => `"${value}"`).join(", ");
+function commaSeparatedString(array = [], prefix = '"', postfix = '"') {
+  return array.map(value => `${prefix}${value}${postfix}`).join(", ");
 }
 
 function formatStringCasing(str, casing, defaultCase) {
@@ -118,6 +118,38 @@ function formatIndexNameForDisplay(index) {
   }
 }
 
+class BatchGetOrderMaintainer {
+  constructor({ table, enabled, keyFormatter }) {
+    this.table = table;
+    this.enabled = enabled;
+    this.keyFormatter = keyFormatter;
+    this.batchIndexMap = new Map();
+    this.currentSlot = 0;
+  }
+
+  getSize() {
+    return this.batchIndexMap.size;
+  }
+
+  getOrder(item) {
+    const key = this.keyFormatter(item);
+    return this.batchIndexMap.get(key) ?? -1;
+  }
+
+  defineOrder(parameters = []) {
+    if (this.enabled) {
+      for (let i = 0; i < parameters.length; i++) {
+        const batchParams = parameters[i];
+        const recordKeys = batchParams?.RequestItems?.[this.table]?.Keys ?? [];
+        for (const recordKey of recordKeys) {
+          const indexMapKey = this.keyFormatter(recordKey);
+          this.batchIndexMap.set(indexMapKey, this.currentSlot++);
+        }
+      }
+    }
+  }
+}
+
 module.exports = {
   batchItems,
   parseJSONPath,
@@ -128,5 +160,6 @@ module.exports = {
   commaSeparatedString,
   formatAttributeCasing,
   applyBetaModelOverrides,
-  formatIndexNameForDisplay
+  formatIndexNameForDisplay,
+  BatchGetOrderMaintainer,
 };

package/src/validations.js

@@ -81,7 +81,6 @@ const Index = {
 				},
 				facets: {
 					type: ["array", "string"],
-					minItems: 1,
 					items: {
 						type: "string",
 					},
@@ -89,7 +88,6 @@ const Index = {
 				},
 				composite: {
 					type: ["array"],
-					minItems: 1,
 					items: {
 						type: "string",
 					},

package/CHANGELOG.md

@@ -1,173 +0,0 @@
-# Changelog
-All notable changes to this project will be documented in this file. Breaking changes to feature functionality will trigger a new Major Version increase. Significant feature improvements and major bug fixes will trigger Minor Version increases. Small, maintenance and additive changes will trigger Patch Version increases.     
-
-## [Unreleased]
-## Changing
-- Bulk Operations to return the original object passed to the operation if that object was returned by DynamoDB as unprocessed.
-
-## [1.0.0] - 2021-06-27
-### Added
-- new `.match()` method to replace original Find method functionality. [[read more]](./README.md#match-records)
-- New `template` property on Model for building custom composite key templates. The `template` property also brings forward a new syntax similar to template literal syntax. [[read more]](./README.md#composite-attribute-templates)
-- New custom composite key syntax using the `template` property. [[read more]](./README.md#composite-attribute-templates)
-- Numeric table keys now possible. Before PK and SK values could only be strings, Numeric Keys are now supported through the `templates` index property. [[read more]](./README.md#numeric-keys) 
-
-### Changed
-- Rename of `facets` property on Model to `composite` for arrays and `template` for string templates. [[read more]](./README.md#the-renaming-of-index-property-facets-to-composite-and-template) 
-- Get method now returns `null` when value is not found. Prior functionality returned an empty object. [[read more]](./README.md#get-record)
-- Strict enforcement of all SK composite attributes being present when performing `.get()`, `.put()`, `.create()`, `.delete()`, `.remove()`, `.update()`, `.patch()` operations.
-- Find method now does not add filters for values supplied, Find now only identifies an Index (if possible) and fulfills the Composite Attributes of that Index (if possible). [[read more]](./README.md#find-records)
-- Query Option `lastEvaluatedKeyRaw` when used with _pagination_ replaced Query Option `pager` with option values: `"raw"`, `"item"`, `"named"`. Default set to `"named"`. [[read more]](./README.md#pager-query-options)  
-- Query Option `lastEvaluatedKeyRaw` when used with _bulk operations_ replaced Query Option `unprocessed` with option values: `"raw"`, `"item"`. Default set to `"item"`. [[read more]](./README.md#query-options)
-
-### Deprecated
-- Removing `facets` property from documentation, examples, and TypeScript typing. Replaced with `composite` property for arrays and `template` for string templates. [[read more]](./README.md#facets)
-- Fully deprecated custom facet string format. Facet strings defined attributes with a prefixed `:` as in `:storeId` would resolve to `storeId`. This has been replaced by the `template` syntax, surrounding the attribute with `${...}`. [[read more]](./README.md#composite-attribute-templates)
-  
-## [1.1.0] - 2021-07-07
-### Added
-- Expanding "collection" concept to include sub-collections. Sub-collections will allow for more precise cross-entity queries to be modeled. [[read more]](./README.md#sub-collections) 
-
-### Fixed
-- Addressed edge-case when modeling sparse indexes that would leave unable to be queried via secondary index. [[read more]](./RELEASE.md#fix-sparse-index-edge-case)
-
-## [1.1.1] - 2021-07-07
-### Added
-- Added new syntax for Attribute Property `watch` to trigger whenever any attribute is updated/retrieved. [[read more]](./README.md#attribute-watching-watch-all)
-
-### Changed
-- The Attribute Property `readOnly` is now enforced _before_ `watch` properties are evaluated. This allows properties that use the Attribute Property `watch` to deliberately circumnavigate `readOnly` enforcement. [[read more]](./README.md#createdat-and-updatedat-attributes)
-  
-## [1.2.0] - 2021-07-31
-### Added
-- Added new update methods `append`, `add`, `subtract`, `data`, `remove`, `delete`, and `data` for improved support of all DynamoDB update methods. [[read more]](./README.md#update-record)
-
-### Changed
-- The property names of `ExpressionAttributeValues` underwent some change in this release due to the addition of new update operations. This is not a breaking change but if you have tests to match on the exact params returned from ElectroDB these will likely break. [[read more]](./RELEASE.md#expressionattributevalues-properties)
-  
-## [1.3.0] - 2021-08-09
-### Added
-- New Attribute types `map`, `list`, `set`. [[read more]](./README.md#expanded-syntax)
-- New Query Options, and support for, `ReturnValues` as requested in Issue#71. [[read more]](./README.md#query-options) 
-- New type definitions for recently released update methods `append`, `add`, `subtract`, `data`, `remove`, and `delete`. [[read more]](./README.md#exported-types) 
-
-### Changed
-- Attributes that have been flagged as `required` are now not possible to be removed (using the update method `remove()`) from a stored Item. This was an oversight from the last release.
-- Attributes that have been flagged as `hidden` now skips invoking that attribute's getter method.  
-
-### Fixed
-- Issues that prevented the nesting of update `value()` operation.
-- TypeScript type definitions for `get()` method now incorporate potential for `null` response.
-- Type definitions for `value()` and `name()` where clause operations.  
-
-## [1.3.1] - 2021-08-09
-### Added
-- New entity method `parse()` to expose ElectroDB formatting for values retrieved outside of ElectroDB. [[read more]](./README.md#parse)
-
-## [1.3.2] - 2021-08-11
-### Fixed
-- Newly added method `parse()` had critical typo. Method now has an improved api, and appropriate tests [[read more]](./README.md#parse)
-
-## [1.4.0] - 2021-08-22
-### Added 
-- Added support for choosing the case ElectroDB will use when modeling a Partition or Sort Key. [[read more]](./README.md#using-electrodb-with-existing-data)
-- Added support for indexes to use fields that are shared with attribute fields. This should help users leverage ElectroDB with existing tables. [[read more]](./README.md#using-electrodb-with-existing-data)
-- Added Query Option `ignoreOwnership` to bypass ElectroDB checks/interrogations for ownership of an item before returning it. [[read more]](./README.md#query-options)
-
-## [1.4.1] - 2021-08-25
-### Added
-- Typedef support for RegExp validation on string attributes
-
-### Fixed
-- RegExp validation issue resulting in undefined (but not required) values being tested.
-
-## [1.4.2] - 2021-09-09
-### Fixed
-- Typing for `.page()` method pager. Now includes the destructured keys associated with the index being queried. [[read more]](./README.md#page)
-- Adding documentation, and expanding typing for the query option `limit`, for use in `.params()` calls. [[read more]](./README.md#query-options)
-
-## [1.4.3] - 2021-10-03
-### Fixed
-- ElectroDB would throw when an `undefined` property was passed to query. This has been changed to not throw if a partial query on that index can be accomplished with the data provided.
-
-## [1.4.4] - 2021-10-16
-### Added
-- Updates did not include composite attributes involved in primary index. Though these values cannot be changed, they should be `set` on update method calls in case the update results in an item insert. [[read more]](./README.md#updates-to-composite-attributes)
-
-## [0.11.1] - 2021-10-17
-### Patched
-- Updates did not include composite attributes involved in primary index. Though these values cannot be changed, they should be `set` on update method calls in case the update results in an item insert. [[read more]](./README.md#updates-to-composite-attributes)
-
-## [1.4.5] - 2021-10-17
-### Fixed
-- Improved .npmignore to remove playground oriented files, and created official directory to keep playground in sync with library changes.
-
-## [1.4.6] - 2021-10-20
-### Added, Fixed
-- Adding Entity identifiers to all update operations. When primary index composite attributes were added in 1.4.4, entities were written properly but did not include the identifiers. This resulted in entities being written but not being readable without the query option `ignoreOwnership` being used.
-
-## [1.4.7] - 2021-10-20
-### Changed
-- Using `add()` update mutation now resolves to `ADD #prop :prop` update expression instead of a `SET #prop = #prop + :prop`
-
-### Fixed
-- Fixed param naming conflict during updates, when map attribute shares a name with another (separate) attribute.
-
-## [1.4.8] - 2021-11-01
-### Fixed
-- Addressed issue#90 to flip batchGet's response tuple type definition.
-
-## [1.5.0] - 2021-11-07
-### Changed
-- Queries will now fully paginate all responses. Prior to this change, ElectroDB would only return items from a single ElectroDB query result. Now ElectroDB will paginate through all query results. This will impact both uses of entity queries and service collections. [[read more](./README.md#query-method)]
-- The query option `limit` has an extended meaning with the change to automatically paginate records on query. The option `limit` now represents a target for the number of items to return from DynamoDB. If this option is passed, Queries on entities and through collections will paginate DynamoDB until this limit is reached or all items for that query have been returned. [[read more](./README.md#query-options)]
-
-### Added
-- A new query option `pages` has been added to coincide with the change to automatically paginate all records when queried. The `pages` option sets a max number of pagination iterations ElectroDB will perform on a query. When this option is paired with `limit`, ElectroDB will respect the first condition reached. [[read more](./README.md#query-options)]
-
-## [1.6.0] - 2021-11-21
-### Added
-- Exporting TypeScript interfaces for `ElectroError` and `ElectroValidationError`
-- Errors thrown within an attribute's validate callback are now wrapped and accessible after being thrown. Prior to this change, only the `message` of the error thrown by a validation function was persisted back through to the user, now the error itself is also accessible. Reference the exported interface typedef for `ElectroValidationError` [here](./index.d.ts) to see the new properties available on a thrown validation error.
-
-### Changed
-- As a byproduct of enhancing validation errors, the format of message text on a validation error has changed. This could be breaking if your app had a hardcoded dependency on the exact text of a thrown validation error.
-
-### Fixed
-- For Set attributes, the callback functions `get`, `set`, and `validate` are now consistently given an Array of values. These functions would sometimes (incorrectly) be called with a DynamoDB DocClient Set.
-
-## [1.6.1] - 2021-12-05
-### Fixed
-- In some cases the `find()` and `match()` methods would incorrectly select an index without a complete partition key. This would result in validation exceptions preventing the user from querying if an index definition and provided attribute object aligned improperly. This was fixed and a slightly more robust mechanism for ranking indexes was made.
-
-## [1.6.2] - 2022-01-27
-### Changed
-- The methods `create`, `patch`, and `remove` will now refer to primary table keys through parameters via ExpressionAttributeNames when using `attribute_exists()`/`attribute_not_exists()` DynamoDB conditions. Prior to this they were referenced directly which would fail in cases where key names include illegal characters. Parameter implementation change only, non-breaking.
-
-## [1.6.3] - 2022-02-22
-### Added
-- Add `data` update operation `ifNotExists` to allow for use of the UpdateExpression function "if_not_exists()".
-
-## [1.7.0] - 2022-03-13
-### Added
-- New feature: "Listeners". Listeners open the door to some really cool tooling that was not possible because of how ElectroDB augments raw DynamoDB responses and did not provide easy access to raw DyanmoDB parameters. [[read more](./README.md#listeners)]
-
-## [1.7.1] - 2022-03-19
-### Added
-- Adding support for the v3 DyanmoDBClient. This change also brings in a new ElectroDB dependency [@aws-sdk/lib-dynamodb](https://www.npmjs.com/package/@aws-sdk/client-dynamodb). [[read more](./README.md#aws-dynamodb-client)]
-
-## [1.7.2] - 2022-03-27
-### Fixed
-- Fixed issue#111, `update` method specific query option typing no longer lost when using a `where` method in a query chain
-- Fixing incorrect typing for exposed `UpdateEntityItem` type. Exported type was missing composite key attributes
-
-## [1.8.0] - 2022-03-28
-### Added
-- Expected typings for the injected v2 client now include methods for `transactWrite` and `transactGet`
-### Changed
-- Map attributes will now always resolve to least an empty object on a `create` and `put` methods (instead of just the root map)
-- In the past, default values for property attributes on maps only resolves when a user provided an object to place the values on. Now default values within maps attributes will now always resolve onto the object on `create` and `put` methods.
-
-## [1.8.1] - 2022-03-29
-### Fixed
-- Solidifying default application methodology: default values for nested properties will be applied up until an undefined default occurs or default callback returns undefined