electrodb 1.6.0 → 1.6.1

package/CHANGELOG.md

@@ -134,4 +134,8 @@ All notable changes to this project will be documented in this file. Breaking ch
 - As a byproduct of enhancing validation errors, the format of message text on a validation error has changed. This could be breaking if your app had a hardcoded dependency on the exact text of a thrown validation error.
 
 ### Fixed
-- For Set attributes, the callback functions `get`, `set`, and `validate` are now consistently given an Array of values. These functions would sometimes (incorrectly) be called with a DynamoDB DocClient Set.
+- For Set attributes, the callback functions `get`, `set`, and `validate` are now consistently given an Array of values. These functions would sometimes (incorrectly) be called with a DynamoDB DocClient Set.
+
+## [1.6.1] - 2021-12-05
+### Fixed
+- In some cases the `find()` and `match()` methods would incorrectly select an index without a complete partition key. This would result in validation exceptions preventing the user from querying if an index definition and provided attribute object aligned improperly. This was fixed and a slightly more robust mechanism for ranking indexes was made.

package/README.md

@@ -569,7 +569,7 @@ const TasksModel = {
 	attributes: {
 		task: {
 			type: "string",
-			default: () => uuidv4(),
+			default: () => uuid(),
 		},
 		project: {
 			type: "string",
@@ -819,11 +819,11 @@ myAttr: {
   watch: ["otherAttr"],
   set: (myAttr, {otherAttr}) => {
     // Whenever "myAttr" or "otherAttr" are updated from an `update` or `patch` operation, this callback will be fired. 
-    // Note: myAttr or otherAttr could be indendently undefined because either attribute could have triggered this callback  
+    // Note: myAttr or otherAttr could be independently undefined because either attribute could have triggered this callback
   },
   get: (myAttr, {otherAttr}) => {
     // Whenever "myAttr" or "otherAttr" are retrieved from a `query` or `get` operation, this callback will be fired. 
-    // Note: myAttr or otherAttr could be indendently undefined because either attribute could have triggered this callback.
+    // Note: myAttr or otherAttr could be independently undefined because either attribute could have triggered this callback.
   } 
 }
 ```
@@ -837,11 +837,11 @@ myAttr: {
   watch: "*", // "watch all"
   set: (myAttr, allAttributes) => {
     // Whenever an `update` or `patch` operation is performed, this callback will be fired. 
-    // Note: myAttr or the attributes under `allAttributes` could be indendently undefined because either attribute could have triggered this callback  
+    // Note: myAttr or the attributes under `allAttributes` could be independently undefined because either attribute could have triggered this callback
   },
   get: (myAttr, allAttributes) => {
     // Whenever a `query` or `get` operation is performed, this callback will be fired. 
-    // Note: myAttr or the attributes under `allAttributes` could be indendently undefined because either attribute could have triggered this callback
+    // Note: myAttr or the attributes under `allAttributes` could be independently undefined because either attribute could have triggered this callback
   } 
 }
 ```
@@ -1421,7 +1421,7 @@ As described in the above two sections ([Composite Attributes](#composite-attrib
 
 It may be the case that an index field is also an attribute. For example, if a table was created with a Primary Index partition key of `accountId`, and that same field is used to store the `accountId` value used by the application. The following are a few examples of how to model that schema with ElectroDB:
 
-> _NOTE: If you have the unique opportunity to use ElectroDB with a new project, it is strongly recommended to use genericly named index fields that are separate from your business attributes._  
+> _NOTE: If you have the unique opportunity to use ElectroDB with a new project, it is strongly recommended to use generically named index fields that are separate from your business attributes._
 
 **Using `composite`**
 
@@ -1762,7 +1762,7 @@ let results = await TaskApp.collections
 
 {
     tasks: [...],    // tasks for employeeId "JExotic" 
-    employees: [...] // employee record(s) with employeeId "JExpotic"
+    employees: [...] // employee record(s) with employeeId "JExotic"
 }
 ```
 
@@ -1777,7 +1777,7 @@ The following is an example of functionally identical collections, implemented a
 **As a string (collection):**
 ```typescript
 {
-  colleciton: "assignments"
+  collection: "assignments"
   pk: {
     field: "pk",
     composite: ["employeeId"]
@@ -1792,7 +1792,7 @@ The following is an example of functionally identical collections, implemented a
 **As a string array (sub-collections):**
 ```typescript
 {
-  colleciton: ["assignments"]
+  collection: ["assignments"]
   pk: {
     field: "pk",
             composite: ["employeeId"]
@@ -3720,6 +3720,8 @@ DynamoDB offers three methods to query records: `get`, `query`, and `scan`. In *
 
 > _NOTE: The Find method is similar to the Match method with one exception: The attributes you supply directly to the `.find()` method will only be used to identify and fulfill your index access patterns. Any values supplied that do not contribute to a composite key will not be applied as query filters. Furthermore, if the values you provide do not resolve to an index access pattern, then a table scan will be performed. Use the `where()` chain method to further filter beyond keys, or use [Match](#match-records) for the convenience of automatic filtering based on the values given directly to that method._
 
+The Find method is useful when the index chosen does not matter or is not known. If your secondary indexes do not contain all attributes then this method might not be right for you. The mechanism that picks the best index for a given payload is subject to improvement and change without triggering a breaking change release version.
+
 ```javascript
 await StoreLocations.find({
     mallId: "EastPointe",
@@ -3750,8 +3752,11 @@ await StoreLocations.find({
 
 Match is a convenience method based off of ElectroDB's [find](#find-records) method. Similar to Find, Match does not require you to provide keys, but under the covers it will leverage the attributes provided to choose the best index to query on.
 
+> _NOTE: The Math method is useful when the index chosen does not matter or is not known. If your secondary indexes do not contain all attributes then this method might not be right for you. The mechanism that picks the best index for a given payload is subject to improvement and change without triggering a breaking change release version.
+
 Match differs from [Find](#find-records) in that it will also include all supplied values into a query filter.
 
+
 ```javascript
 await StoreLocations.find({
     mallId: "EastPointe",
@@ -4228,7 +4233,7 @@ await StoreLocations.query.leases({storeId}).gte({leaseEndDate: "2020-03"}).go()
 // Lease Agreements by StoreId before 2021
 await StoreLocations.query.leases({storeId}).lt({leaseEndDate: "2021-01"}).go()
 
-// Lease Agreements by StoreId before Feburary 2021
+// Lease Agreements by StoreId before February 2021
 await StoreLocations.query.leases({storeId}).lte({leaseEndDate: "2021-02"}).go()
 
 // Lease Agreements by StoreId between 2010 and 2020
@@ -4589,6 +4594,15 @@ Below is the type definition for an ElectroValidationError:
 ```typescript
 ElectroValidationError<T extends Error = Error> extends ElectroError {
     readonly name: "ElectroValidationError"
+    readonly code: number;
+    readonly date: number;
+    readonly isElectroError: boolean;
+    ref: {
+        readonly code: number;
+        readonly section: string;
+        readonly name: string;
+        readonly sym: unique symbol;
+    }
     readonly fields: ReadonlyArray<{
         /**
          * The json path to the attribute that had a validation error

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "electrodb",
-  "version": "1.6.0",
+  "version": "1.6.1",
   "description": "A library to more easily create and interact with multiple entities and heretical relationships in dynamodb",
   "main": "index.js",
   "scripts": {
@@ -25,7 +25,7 @@
   },
   "homepage": "https://github.com/tywalch/electrodb#readme",
   "devDependencies": {
-    "@istanbuljs/nyc-config-typescript": "^1.0.1",
+    "@istanbuljs/nyc-config-typescript": "^1.0.2",
     "@types/chai": "^4.2.12",
     "@types/mocha": "^8.0.3",
     "@types/node": "^15.6.0",
@@ -33,7 +33,7 @@
     "aws-sdk": "2.630.0",
     "browserify": "^17.0.0",
     "chai": "4.2.0",
-    "coveralls": "^3.1.0",
+    "coveralls": "^3.1.1",
     "istanbul": "0.4.5",
     "jest": "25.4.0",
     "mocha": "7.1.1",
@@ -50,6 +50,9 @@
     "electrodb",
     "dynamo",
     "dynamodb",
+    "nosql",
+    "single table design",
+    "typescript",
     "aws"
   ],
   "tsd": {

package/src/entity.js

@@ -1885,54 +1885,105 @@ class Entity {
 		return utilities.formatKeyCasing(key, casing);
 	}
 
-	_findBestIndexKeyMatch(attributes) {
-		let candidates = this.model.facets.bySlot.map((val, i) => i);
+	_findBestIndexKeyMatch(attributes = {}) {
+		// an array of arrays, representing the order of pk and sk composites specified for each index, and then an
+		// array with each access pattern occupying the same array index.
 		let facets = this.model.facets.bySlot;
-		let match;
-		let keys = {};
-		for (let i = 0; i < facets.length; i++) {
-			let currentMatches = [];
-			let nextMatches = [];
-			for (let j = 0; j < candidates.length; j++) {
-				let slot = candidates[j];
-				if (!facets[i][slot]) {
-					continue;
-				}
-				let name = facets[i][slot].name;
-				let next = facets[i][slot].next;
-				let index = facets[i][slot].index;
-				let type = facets[i][slot].type;
-				let match = !!attributes[name];
-				let matchNext = !!attributes[next];
-				if (match) {
-					keys[index] = keys[index] || [];
-					keys[index].push({ name, type });
-					currentMatches.push(slot);
-					if (matchNext) {
-						nextMatches.push(slot);
-					}
+		// a flat array containing the match results of each access pattern, in the same array index they occur within
+		// bySlot above
+		let matches = [];
+		for (let f = 0; f < facets.length; f++) {
+			const slots = facets[f] || [];
+			for (let s = 0; s < slots.length; s++) {
+				const accessPatternSlot = slots[s];
+				matches[s] = matches[s] || {
+					index: accessPatternSlot.index,
+					allKeys: false,
+					hasSk: false,
+					count: 0,
+					done: false,
+					keys: []
 				}
-			}
-			if (currentMatches.length) {
-				if (nextMatches.length) {
-					candidates = [...nextMatches];
+				// already determined to be out of contention on prior iteration
+				const indexOutOfContention = matches[s].done;
+				// composite shorter than other indexes
+				const lacksAttributeAtSlot = !accessPatternSlot;
+				// attribute at this slot is not in the object provided
+				const attributeNotProvided = accessPatternSlot && attributes[accessPatternSlot.name] === undefined;
+				// if the next attribute is a sort key then all partition keys were provided
+				const nextAttributeIsSortKey = accessPatternSlot && accessPatternSlot.next && facets[f+1][s].type === "sk";
+				// if no keys are left then all attribute requirements were met (remember indexes don't require a sort key)
+				const hasAllKeys = accessPatternSlot && !accessPatternSlot.next;
+
+				// no sense iterating on items we know to be "done"
+				if (indexOutOfContention || lacksAttributeAtSlot || attributeNotProvided) {
+					matches[s].done = true;
 					continue;
-				} else {
-					match = facets[i][currentMatches[0]].index;
-					break;
 				}
-			} else if (i === 0) {
-				break;
-			} else {
-				match = (candidates[0] !== undefined && facets[i][candidates[0]].index) || TableIndex;
-				break;
+
+				// if the next attribute is a sort key (and you reached this line) then you have fulfilled all the
+				// partition key requirements for this index
+				if (nextAttributeIsSortKey) {
+					matches[s].hasSk = true;
+				// if you reached this step and there are no more attributes, then you fulfilled the index
+				} else if (hasAllKeys) {
+					matches[s].allKeys = true;
+				}
+
+				// number of successfully fulfilled attributes plays into the ranking heuristic
+				matches[s].count++;
+
+				// note the names/types of fulfilled attributes
+				matches[s].keys.push({
+					name: accessPatternSlot.name,
+					type: accessPatternSlot.type
+				});
 			}
 		}
-		return {
-			keys: keys[match] || [],
-			index: match || TableIndex,
-			shouldScan: match === undefined
-		};
+		// the highest count of matched attributes among all access patterns
+		let max = 0;
+		matches = matches
+			// remove incomplete indexes
+			.filter(match => match.hasSk || match.allKeys)
+			// calculate max attribute match
+			.map(match => {
+				max = Math.max(max, match.count);
+				return match;
+			});
+
+		// matched contains the ranked attributes. The closer an element is to zero the "higher" the rank.
+		const matched = [];
+		for (let m = 0; m < matches.length; m++) {
+			const match = matches[m];
+			// a finished primary index is most ideal (could be a get)
+			const primaryIndexIsFinished = match.index === "" && match.allKeys;
+			// if there is a tie for matched index attributes, primary index should win
+			const primaryIndexIsMostMatched = match.index === "" && match.count === max;
+			// composite attributes are complete
+			const indexRequirementsFulfilled = match.allKeys;
+			// having the most matches is important
+			const hasTheMostAttributeMatches = match.count === max;
+			if (primaryIndexIsFinished) {
+				matched[0] = match;
+			} else if (primaryIndexIsMostMatched) {
+				matched[1] = match;
+			} else if (indexRequirementsFulfilled) {
+				matched[2] = match;
+			} else if (hasTheMostAttributeMatches) {
+				matched[3] = match;
+			}
+		}
+		// find the first non-undefined element (best ranked) -- if possible
+		const match = matched.find(value => !!value);
+		let keys = [];
+		let index = "";
+		let shouldScan = true;
+		if (match) {
+			keys = match.keys;
+			index = match.index;
+			shouldScan = false;
+		}
+		return { keys, index, shouldScan };
 	}
 
 	/* istanbul ignore next */
@@ -1969,7 +2020,7 @@ class Entity {
 				type = "name";
 			} else if (char === "}" && type === "name") {
 				if (current.name.match(/^\s*$/)) {
-					throw new e.ElectroError(e.ErrorCodes.InvalidKeyCompositeAttributeTemplate, `Invalid key composite attribute template. Empty expression "\${${current.name}" provided. Expected attribute name.`);
+					throw new e.ElectroError(e.ErrorCodes.InvalidKeyCompositeAttributeTemplate, `Invalid key composite attribute template. Empty expression "\${${current.name}}" provided. Expected attribute name.`);
 				}
 				attributes.push({name: current.name, label: current.label});
 				current.name = "";