electrodb 1.5.0 → 1.6.3

package/CHANGELOG.md

@@ -98,28 +98,52 @@ All notable changes to this project will be documented in this file. Breaking ch
 ### Patched
 - Updates did not include composite attributes involved in primary index. Though these values cannot be changed, they should be `set` on update method calls in case the update results in an item insert. [[read more]](./README.md#updates-to-composite-attributes)
 
-## [1.4.5] = 2021-10-17
+## [1.4.5] - 2021-10-17
 ### Fixed
 - Improved .npmignore to remove playground oriented files, and created official directory to keep playground in sync with library changes.
 
-## [1.4.6] = 2021-10-20
+## [1.4.6] - 2021-10-20
 ### Added, Fixed
 - Adding Entity identifiers to all update operations. When primary index composite attributes were added in 1.4.4, entities were written properly but did not include the identifiers. This resulted in entities being written but not being readable without the query option `ignoreOwnership` being used.
 
-## [1.4.7] = 2021-10-20
+## [1.4.7] - 2021-10-20
 ### Changed
 - Using `add()` update mutation now resolves to `ADD #prop :prop` update expression instead of a `SET #prop = #prop + :prop`
 
 ### Fixed
 - Fixed param naming conflict during updates, when map attribute shares a name with another (separate) attribute.
 
-## [1.4.8] = 2021-11-01
+## [1.4.8] - 2021-11-01
 ### Fixed
 - Addressed issue#90 to flip batchGet's response tuple type definition.
 
-## [1.5.0] = 2021-11-07
+## [1.5.0] - 2021-11-07
 ### Changed
 - Queries will now fully paginate all responses. Prior to this change, ElectroDB would only return items from a single ElectroDB query result. Now ElectroDB will paginate through all query results. This will impact both uses of entity queries and service collections. [[read more](./README.md#query-method)]
 - The query option `limit` has an extended meaning with the change to automatically paginate records on query. The option `limit` now represents a target for the number of items to return from DynamoDB. If this option is passed, Queries on entities and through collections will paginate DynamoDB until this limit is reached or all items for that query have been returned. [[read more](./README.md#query-options)]
+
+### Added
+- A new query option `pages` has been added to coincide with the change to automatically paginate all records when queried. The `pages` option sets a max number of pagination iterations ElectroDB will perform on a query. When this option is paired with `limit`, ElectroDB will respect the first condition reached. [[read more](./README.md#query-options)]
+
+## [1.6.0] - 2021-11-21
+### Added
+- Exporting TypeScript interfaces for `ElectroError` and `ElectroValidationError`
+- Errors thrown within an attribute's validate callback are now wrapped and accessible after being thrown. Prior to this change, only the `message` of the error thrown by a validation function was persisted back through to the user, now the error itself is also accessible. Reference the exported interface typedef for `ElectroValidationError` [here](./index.d.ts) to see the new properties available on a thrown validation error.
+
+### Changed
+- As a byproduct of enhancing validation errors, the format of message text on a validation error has changed. This could be breaking if your app had a hardcoded dependency on the exact text of a thrown validation error.
+
+### Fixed
+- For Set attributes, the callback functions `get`, `set`, and `validate` are now consistently given an Array of values. These functions would sometimes (incorrectly) be called with a DynamoDB DocClient Set.
+
+## [1.6.1] - 2021-12-05
+### Fixed
+- In some cases the `find()` and `match()` methods would incorrectly select an index without a complete partition key. This would result in validation exceptions preventing the user from querying if an index definition and provided attribute object aligned improperly. This was fixed and a slightly more robust mechanism for ranking indexes was made.
+
+## [1.6.2] - 2022-01-27
+### Changed
+- The methods `create`, `patch`, and `remove` will now refer to primary table keys through parameters via ExpressionAttributeNames when using `attribute_exists()`/`attribute_not_exists()` DynamoDB conditions. Prior to this they were referenced directly which would fail in cases where key names include illegal characters. Parameter implementation change only, non-breaking.
+
+## [1.6.3] - 2022-02-22
 ### Added
-- A new query option `pages` has been added to coincide with the change to automatically paginate all records when queried. The `pages` option sets a max number of pagination iterations ElectroDB will perform on a query. When this option is paired with `limit`, ElectroDB will respect the first condition reached. [[read more](./README.md#query-options)]
+- Add `data` update operation `ifNotExists` to allow for use of the UpdateExpression function "if_not_exists()".

package/README.md

@@ -2,7 +2,7 @@
 [![Coverage Status](https://coveralls.io/repos/github/tywalch/electrodb/badge.svg?branch=master)](https://coveralls.io/github/tywalch/electrodb?branch=master&kill_cache=please)
 [![Coverage Status](https://img.shields.io/npm/dt/electrodb.svg)](https://www.npmjs.com/package/electrodb)
 ![npm bundle size](https://img.shields.io/bundlephobia/min/electrodb) [![Build Status](https://travis-ci.org/tywalch/electrodb.svg?branch=master)](https://travis-ci.org/tywalch/electrodb)
-[![Runkit Demo](https://img.shields.io/badge/runkit-electrodb-db4792)](https://runkit.com/tywalch/creating-and-querying-an-electrodb-service)
+[![Runkit Demo](https://img.shields.io/badge/runkit-electrodb-db4792)](https://runkit.com/tywalch/electrodb-building-queries)
 
 ![ElectroDB](https://github.com/tywalch/electrodb/blob/master/assets/electrodb-drk.png?raw=true)
 ***ElectroDB*** is a DynamoDB library to ease the use of having multiple entities and complex hierarchical relationships in a single DynamoDB table.
@@ -107,11 +107,11 @@ tasks
   * [TypeScript Support](#typescript-support)
     + [TypeScript Services](#typescript-services)
   * [Join](#join)
-    - [Independent Models](#independent-models)
-    - [Joining Entity instances to a Service](#joining-entity-instances-to-a-service)
-    - [Joining models to a Service](#joining-models-to-a-service)
-    - [Joining Entities or Models with an alias](#joining-entities-or-models-with-an-alias)
-    - [Joining Entities at Service construction for TypeScript](#joining-entities-at-service-construction-for-typescript)
+      - [Independent Models](#independent-models)
+      - [Joining Entity instances to a Service](#joining-entity-instances-to-a-service)
+      - [Joining models to a Service](#joining-models-to-a-service)
+      - [Joining Entities or Models with an alias](#joining-entities-or-models-with-an-alias)
+      - [Joining Entities at Service construction for TypeScript](#joining-entities-at-service-construction-for-typescript)
   * [Model](#model)
     + [Model Properties](#model-properties)
     + [Service Options](#service-options)
@@ -125,7 +125,7 @@ tasks
       - [Set Attributes](#set-attributes)
       - [Attribute Getters and Setters](#attribute-getters-and-setters)
       - [Attribute Watching](#attribute-watching)
-        * [Attribute Watching: Watch All](#attribute-watching-watch-all)
+        * [Attribute Watching: Watch All](#attribute-watching--watch-all)
         * [Attribute Watching Examples](#attribute-watching-examples)
       - [Calculated Attributes](#calculated-attributes)
       - [Virtual Attributes](#virtual-attributes)
@@ -147,7 +147,7 @@ tasks
     + [Collection Queries vs Entity Queries](#collection-queries-vs-entity-queries)
     + [Collection Response Structure](#collection-response-structure)
   * [Sub-Collections](#sub-collections)
-    - [Sub-Collection Entities](#sub-collection-entities)
+      - [Sub-Collection Entities](#sub-collection-entities)
   * [Index and Collection Naming Conventions](#index-and-collection-naming-conventions)
     + [Index Naming Conventions](#index-naming-conventions)
   * [Collection Naming Conventions](#collection-naming-conventions)
@@ -163,11 +163,11 @@ tasks
     + [Multiple Where Clauses](#multiple-where-clauses)
   * [Parse](#parse)
 - [Building Queries](#building-queries)
-  + [Using composite attributes to make hierarchical keys](#using-composite-attributes-to-make-hierarchical-keys)
-    - [Shopping Mall Stores](#shopping-mall-stores)
-  + [Query App Records](#query-app-records)
-    - [Partition Key Composite Attributes](#partition-key-composite-attributes)
-  + [Sort Key Operations](#sort-key-operations)
+    + [Using composite attributes to make hierarchical keys](#using-composite-attributes-to-make-hierarchical-keys)
+      - [Shopping Mall Stores](#shopping-mall-stores)
+    + [Query App Records](#query-app-records)
+      - [Partition Key Composite Attributes](#partition-key-composite-attributes)
+    + [Sort Key Operations](#sort-key-operations)
   * [Query Chains](#query-chains)
     + [Query Method](#query-method)
     + [Get Method](#get-method)
@@ -178,14 +178,14 @@ tasks
     + [Batch Write Put Records](#batch-write-put-records)
     + [Update Record](#update-record)
       - [Updates to Composite Attributes](#updates-to-composite-attributes)
-      - [Update Method: Set](#update-method-set)
-      - [Update Method: Remove](#update-method-remove)
-      - [Update Method: Add](#update-method-add)
-      - [Update Method: Subtract](#update-method-subtract)
-      - [Update Method: Append](#update-method-append)
-      - [Update Method: Delete](#update-method-delete)
-      - [Update Method: Data](#update-method-data)
-    + [Update Method: Complex Data Types](#update-method-complex-data-types)
+      - [Update Method: Set](#update-method--set)
+      - [Update Method: Remove](#update-method--remove)
+      - [Update Method: Add](#update-method--add)
+      - [Update Method: Subtract](#update-method--subtract)
+      - [Update Method: Append](#update-method--append)
+      - [Update Method: Delete](#update-method--delete)
+      - [Update Method: Data](#update-method--data)
+    + [Update Method: Complex Data Types](#update-method--complex-data-types)
     + [Scan Records](#scan-records)
     + [Remove Method](#remove-method)
     + [Patch Record](#patch-record)
@@ -205,30 +205,33 @@ tasks
         * [Pagination Example](#pagination-example)
   * [Query Examples](#query-examples)
   * [Query Options](#query-options)
-- [Errors:](#errors)
-  + [No Client Defined On Model](#no-client-defined-on-model)
-  + [Invalid Identifier](#invalid-identifier)
-  + [Invalid Key Composite Attribute Template](#invalid-key-composite-attribute-template)
-  + [Duplicate Indexes](#duplicate-indexes)
-  + [Collection Without An SK](#collection-without-an-sk)
-  + [Duplicate Collections](#duplicate-collections)
-  + [Missing Primary Index](#missing-primary-index)
-  + [Invalid Attribute Definition](#invalid-attribute-definition)
-  + [Invalid Model](#invalid-model)
-  + [Invalid Options](#invalid-options)
-  + [Duplicate Index Fields](#duplicate-index-fields)
-  + [Duplicate Index Composite Attributes](#duplicate-index-composite-attributes)
-  + [Incompatible Key Composite Attribute Template](#incompatible-key-composite-attribute-template)
-  + [Invalid Index With Attribute Name](#invalid-index-with-attribute-name)
-  + [Invalid Collection on Index With Attribute Field Names](#invalid-collection-on-index-with-attribute-field-names)
-  + [Missing Composite Attributes](#missing-composite-attributes)
-  + [Missing Table](#missing-table)
-  + [Invalid Concurrency Option](#invalid-concurrency-option)
-  + [aws-error](#aws-error)
-  + [Unknown Errors](#unknown-errors)
-  + [Invalid Last Evaluated Key](#invalid-last-evaluated-key)
-  + [No Owner For Pager](#no-owner-for-pager)
-  + [Pager Not Unique](#pager-not-unique)
+- [Errors:](#errors-)
+    + [No Client Defined On Model](#no-client-defined-on-model)
+    + [Invalid Identifier](#invalid-identifier)
+    + [Invalid Key Composite Attribute Template](#invalid-key-composite-attribute-template)
+    + [Duplicate Indexes](#duplicate-indexes)
+    + [Collection Without An SK](#collection-without-an-sk)
+    + [Duplicate Collections](#duplicate-collections)
+    + [Missing Primary Index](#missing-primary-index)
+    + [Invalid Attribute Definition](#invalid-attribute-definition)
+    + [Invalid Model](#invalid-model)
+    + [Invalid Options](#invalid-options)
+    + [Duplicate Index Fields](#duplicate-index-fields)
+    + [Duplicate Index Composite Attributes](#duplicate-index-composite-attributes)
+    + [Incompatible Key Composite Attribute Template](#incompatible-key-composite-attribute-template)
+    + [Invalid Index With Attribute Name](#invalid-index-with-attribute-name)
+    + [Invalid Collection on Index With Attribute Field Names](#invalid-collection-on-index-with-attribute-field-names)
+    + [Missing Composite Attributes](#missing-composite-attributes)
+    + [Missing Table](#missing-table)
+    + [Invalid Concurrency Option](#invalid-concurrency-option)
+    + [Invalid Pages Option](#invalid-pages-option)
+    + [Invalid Limit Option](#invalid-limit-option)
+    + [Invalid Attribute](#invalid-attribute)
+    + [AWS Error](#aws-error)
+    + [Unknown Errors](#unknown-errors)
+    + [Invalid Last Evaluated Key](#invalid-last-evaluated-key)
+    + [No Owner For Pager](#no-owner-for-pager)
+    + [Pager Not Unique](#pager-not-unique)
 - [Examples](#examples)
   * [Employee App](#employee-app)
     + [Employee App Requirements](#employee-app-requirements)
@@ -566,7 +569,7 @@ const TasksModel = {
 	attributes: {
 		task: {
 			type: "string",
-			default: () => uuidv4(),
+			default: () => uuid(),
 		},
 		project: {
 			type: "string",
@@ -816,11 +819,11 @@ myAttr: {
   watch: ["otherAttr"],
   set: (myAttr, {otherAttr}) => {
     // Whenever "myAttr" or "otherAttr" are updated from an `update` or `patch` operation, this callback will be fired. 
-    // Note: myAttr or otherAttr could be indendently undefined because either attribute could have triggered this callback  
+    // Note: myAttr or otherAttr could be independently undefined because either attribute could have triggered this callback
   },
   get: (myAttr, {otherAttr}) => {
     // Whenever "myAttr" or "otherAttr" are retrieved from a `query` or `get` operation, this callback will be fired. 
-    // Note: myAttr or otherAttr could be indendently undefined because either attribute could have triggered this callback.
+    // Note: myAttr or otherAttr could be independently undefined because either attribute could have triggered this callback.
   } 
 }
 ```
@@ -834,11 +837,11 @@ myAttr: {
   watch: "*", // "watch all"
   set: (myAttr, allAttributes) => {
     // Whenever an `update` or `patch` operation is performed, this callback will be fired. 
-    // Note: myAttr or the attributes under `allAttributes` could be indendently undefined because either attribute could have triggered this callback  
+    // Note: myAttr or the attributes under `allAttributes` could be independently undefined because either attribute could have triggered this callback
   },
   get: (myAttr, allAttributes) => {
     // Whenever a `query` or `get` operation is performed, this callback will be fired. 
-    // Note: myAttr or the attributes under `allAttributes` could be indendently undefined because either attribute could have triggered this callback
+    // Note: myAttr or the attributes under `allAttributes` could be independently undefined because either attribute could have triggered this callback
   } 
 }
 ```
@@ -1418,7 +1421,7 @@ As described in the above two sections ([Composite Attributes](#composite-attrib
 
 It may be the case that an index field is also an attribute. For example, if a table was created with a Primary Index partition key of `accountId`, and that same field is used to store the `accountId` value used by the application. The following are a few examples of how to model that schema with ElectroDB:
 
-> _NOTE: If you have the unique opportunity to use ElectroDB with a new project, it is strongly recommended to use genericly named index fields that are separate from your business attributes._  
+> _NOTE: If you have the unique opportunity to use ElectroDB with a new project, it is strongly recommended to use generically named index fields that are separate from your business attributes._
 
 **Using `composite`**
 
@@ -1759,7 +1762,7 @@ let results = await TaskApp.collections
 
 {
     tasks: [...],    // tasks for employeeId "JExotic" 
-    employees: [...] // employee record(s) with employeeId "JExpotic"
+    employees: [...] // employee record(s) with employeeId "JExotic"
 }
 ```
 
@@ -1774,7 +1777,7 @@ The following is an example of functionally identical collections, implemented a
 **As a string (collection):**
 ```typescript
 {
-  colleciton: "assignments"
+  collection: "assignments"
   pk: {
     field: "pk",
     composite: ["employeeId"]
@@ -1789,7 +1792,7 @@ The following is an example of functionally identical collections, implemented a
 **As a string array (sub-collections):**
 ```typescript
 {
-  colleciton: ["assignments"]
+  collection: ["assignments"]
   pk: {
     field: "pk",
             composite: ["employeeId"]
@@ -3459,7 +3462,8 @@ operation     | example                               | result
 `delete`      | `delete(tenant, name)`                | `#tenant :tenant1`                                                    | Remove item from existing `set` attribute
 `del`         | `del(tenant, name)`                   | `#tenant :tenant1`                                                    | Alias for `delete` operation
 `name`        | `name(rent)`                          | `#rent`                                                               | Reference another attribute's name, can be passed to other operation that allows leveraging existing attribute values in calculating new values
-`value`       | `value(rent, value)`                  | `:rent1`                                                              | Create a reference to a particular value, can be passed to other operation that allows leveraging existing attribute values in calculating new values
+`value`       | `value(rent, amount)`                 | `:rent1`                                                              | Create a reference to a particular value, can be passed to other operation that allows leveraging existing attribute values in calculating new values
+`ifNotExists` | `ifNotExists(rent, amount)`           | `#rent = if_not_exists(#rent, :rent0)`                                | Update a property's value only if that property doesn't yet exist on the record
 
 ```javascript
 await StoreLocations
@@ -3717,6 +3721,8 @@ DynamoDB offers three methods to query records: `get`, `query`, and `scan`. In *
 
 > _NOTE: The Find method is similar to the Match method with one exception: The attributes you supply directly to the `.find()` method will only be used to identify and fulfill your index access patterns. Any values supplied that do not contribute to a composite key will not be applied as query filters. Furthermore, if the values you provide do not resolve to an index access pattern, then a table scan will be performed. Use the `where()` chain method to further filter beyond keys, or use [Match](#match-records) for the convenience of automatic filtering based on the values given directly to that method._
 
+The Find method is useful when the index chosen does not matter or is not known. If your secondary indexes do not contain all attributes then this method might not be right for you. The mechanism that picks the best index for a given payload is subject to improvement and change without triggering a breaking change release version.
+
 ```javascript
 await StoreLocations.find({
     mallId: "EastPointe",
@@ -3747,8 +3753,11 @@ await StoreLocations.find({
 
 Match is a convenience method based off of ElectroDB's [find](#find-records) method. Similar to Find, Match does not require you to provide keys, but under the covers it will leverage the attributes provided to choose the best index to query on.
 
+> _NOTE: The Math method is useful when the index chosen does not matter or is not known. If your secondary indexes do not contain all attributes then this method might not be right for you. The mechanism that picks the best index for a given payload is subject to improvement and change without triggering a breaking change release version.
+
 Match differs from [Find](#find-records) in that it will also include all supplied values into a query filter.
 
+
 ```javascript
 await StoreLocations.find({
     mallId: "EastPointe",
@@ -4225,7 +4234,7 @@ await StoreLocations.query.leases({storeId}).gte({leaseEndDate: "2020-03"}).go()
 // Lease Agreements by StoreId before 2021
 await StoreLocations.query.leases({storeId}).lt({leaseEndDate: "2021-01"}).go()
 
-// Lease Agreements by StoreId before Feburary 2021
+// Lease Agreements by StoreId before February 2021
 await StoreLocations.query.leases({storeId}).lte({leaseEndDate: "2021-02"}).go()
 
 // Lease Agreements by StoreId between 2010 and 2020
@@ -4572,7 +4581,54 @@ When performing a query [Query](#building-queries) you can pass a [Query Options
 *What to do about it:*
 Expect this error only if you're providing a `limit` option. Double-check the value you are providing is the value you expect to be passing, and that the value passes the tests listed above.
 
-### aws-error
+### Invalid Attribute
+*Code: 3001*
+
+*Why this occurred:*
+The value received for a validation either failed type expectations (e.g. a "number" instead of a "string"), or the user provided "validate" callback on an attribute rejected a value.
+
+*What to do about it:*
+Examine the error itself for more precise detail on why the failure occurred. The error object itself should have a property called "fields" which contains an array of every attribute that failed validation, and a reason for each. If the failure originated from a "validate" callback, the originally thrown error will be accessible via the `cause` property the corresponding element within the fields array.1
+
+Below is the type definition for an ElectroValidationError:
+
+```typescript
+ElectroValidationError<T extends Error = Error> extends ElectroError {
+    readonly name: "ElectroValidationError"
+    readonly code: number;
+    readonly date: number;
+    readonly isElectroError: boolean;
+    ref: {
+        readonly code: number;
+        readonly section: string;
+        readonly name: string;
+        readonly sym: unique symbol;
+    }
+    readonly fields: ReadonlyArray<{
+        /**
+         * The json path to the attribute that had a validation error
+         */
+        readonly field: string;
+
+        /**
+         * A description of the validation error for that attribute
+         */
+        readonly reason: string;
+
+        /**
+         * Index of the value passed (present only in List attribute validation errors)
+         */
+        readonly index: number | undefined;
+
+        /**
+         * The error thrown from the attribute's validate callback (if applicable)
+         */
+        readonly cause: T | undefined;
+    }>
+}
+```
+
+### AWS Error
 *Code: 4001*
 
 *Why this occurred:*
@@ -5497,4 +5553,4 @@ This change stems from the fact the `facets` is already a defined term in the Dy
 1.0.0 brings back a `null` response from the `get()` method when a record could not be found. Prior to `1.0.0` ElectroDB returned an empty object.
 
 # Coming Soon
-- Default query options defined on the `model` to give more general control of interactions with the Entity.
+- Default query options defined on the `model` to give more general control of interactions with the Entity.

package/index.d.ts

@@ -1,6 +1,45 @@
 declare const WhereSymbol: unique symbol;
 declare const UpdateDataSymbol: unique symbol;
 
+export interface ElectroError extends Error {
+    readonly name: 'ElectroError';
+    readonly code: number;
+    readonly date: number;
+    readonly isElectroError: boolean;
+    ref: {
+        readonly code: number;
+        readonly section: string;
+        readonly name: string;
+        readonly sym: unique symbol;
+    }
+}
+
+interface ElectroValidationErrorFieldReference<T extends Error = Error> {
+    /**
+     * The json path to the attribute that had a validation error
+     */
+    readonly field: string;
+
+    /**
+     * A description of the validation error for that attribute
+     */
+    readonly reason: string;
+
+    /**
+     * Index of the value passed (present only in List attribute validation errors)
+     */
+    readonly index: number | undefined;
+
+    /**
+     * The error thrown from the attribute's validate callback (if applicable)
+     */
+    readonly cause: T | undefined;
+}
+
+export interface ElectroValidationError<T extends Error = Error> extends ElectroError {
+    readonly fields: ReadonlyArray<ElectroValidationErrorFieldReference<T>>;
+}
+
 interface ReadOnlyAttribute {
     readonly readOnly: true;
 }
@@ -915,6 +954,7 @@ type DataUpdateOperations<A extends string, F extends A, C extends string, S ext
     del:    <T, A extends DataUpdateAttributeSymbol<T>>(attr: A, value: A extends DataUpdateAttributeSymbol<infer V> ? V extends Array<any> ? V : never : never ) => any;
     value: <T, A extends DataUpdateAttributeSymbol<T>>(attr: A, value: DataUpdateAttributeValues<A>) => Required<DataUpdateAttributeValues<A>>;
     name: <T, A extends DataUpdateAttributeSymbol<T>>(attr: A) => any;
+    ifNotExists: <T, A extends DataUpdateAttributeSymbol<T>>(attr: A, value: DataUpdateAttributeValues<A>) => any;
 };
 
 type WhereCallback<A extends string, F extends A, C extends string, S extends Schema<A,F,C>, I extends Item<A,F,C,S,S["attributes"]>> =

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "electrodb",
-  "version": "1.5.0",
+  "version": "1.6.3",
   "description": "A library to more easily create and interact with multiple entities and heretical relationships in dynamodb",
   "main": "index.js",
   "scripts": {
@@ -25,7 +25,7 @@
   },
   "homepage": "https://github.com/tywalch/electrodb#readme",
   "devDependencies": {
-    "@istanbuljs/nyc-config-typescript": "^1.0.1",
+    "@istanbuljs/nyc-config-typescript": "^1.0.2",
     "@types/chai": "^4.2.12",
     "@types/mocha": "^8.0.3",
     "@types/node": "^15.6.0",
@@ -33,7 +33,7 @@
     "aws-sdk": "2.630.0",
     "browserify": "^17.0.0",
     "chai": "4.2.0",
-    "coveralls": "^3.1.0",
+    "coveralls": "^3.1.1",
     "istanbul": "0.4.5",
     "jest": "25.4.0",
     "mocha": "7.1.1",
@@ -50,6 +50,9 @@
     "electrodb",
     "dynamo",
     "dynamodb",
+    "nosql",
+    "single table design",
+    "typescript",
     "aws"
   ],
   "tsd": {

package/src/clauses.js

@@ -1,5 +1,5 @@
-const { QueryTypes, MethodTypes, ItemOperations, ExpressionTypes } = require("./types");
-const {AttributeOperationProxy, UpdateOperations} = require("./operations");
+const { QueryTypes, MethodTypes, ItemOperations, ExpressionTypes, TableIndex } = require("./types");
+const {AttributeOperationProxy, UpdateOperations, FilterOperationNames} = require("./operations");
 const {UpdateExpression} = require("./update");
 const {FilterExpression} = require("./where");
 const v = require("./validations");
@@ -134,6 +134,12 @@ let clauses = {
 			}
 			try {
 				const attributes = state.getCompositeAttributes();
+				const filter = state.query.filter[ExpressionTypes.ConditionExpression];
+				const {pk, sk} = entity._getPrimaryIndexFieldNames();
+				filter.unsafeSet(FilterOperationNames.exists, pk);
+				if (sk) {
+					filter.unsafeSet(FilterOperationNames.exists, sk);
+				}
 				return state
 					.setMethod(MethodTypes.delete)
 					.setType(QueryTypes.eq)
@@ -189,6 +195,12 @@ let clauses = {
 			try {
 				let record = entity.model.schema.checkCreate({...payload});
 				const attributes = state.getCompositeAttributes();
+				const filter = state.query.filter[ExpressionTypes.ConditionExpression];
+				const {pk, sk} = entity._getPrimaryIndexFieldNames();
+				filter.unsafeSet(FilterOperationNames.notExists, pk);
+				if (sk) {
+					filter.unsafeSet(FilterOperationNames.notExists, sk);
+				}
 				return state
 					.setMethod(MethodTypes.put)
 					.setType(QueryTypes.eq)
@@ -213,6 +225,12 @@ let clauses = {
 			}
 			try {
 				const attributes = state.getCompositeAttributes();
+				const filter = state.query.filter[ExpressionTypes.ConditionExpression];
+				const {pk, sk} = entity._getPrimaryIndexFieldNames();
+				filter.unsafeSet(FilterOperationNames.exists, pk);
+				if (sk) {
+					filter.unsafeSet(FilterOperationNames.exists, sk);
+				}
 				return state
 					.setMethod(MethodTypes.update)
 					.setType(QueryTypes.eq)

package/src/entity.js

@@ -198,11 +198,7 @@ class Entity {
 
 	create(attributes = {}) {
 		let index = TableIndex;
-		let options = {
-			params: {
-				ConditionExpression: this._makeItemDoesntExistConditions(index)
-			}
-		};
+		let options = {};
 		return this._makeChain(index, this._clausesWithFilters, clauses.index, options).create(attributes);
 	}
 
@@ -213,21 +209,13 @@ class Entity {
 
 	patch(facets = {}) {
 		let index = TableIndex;
-		let options = {
-			params: {
-				ConditionExpression: this._makeItemExistsConditions(index)
-			}
-		};
+		let options = {};
 		return this._makeChain(index, this._clausesWithFilters, clauses.index, options).patch(facets);
 	}
 
 	remove(facets = {}) {
 		let index = TableIndex;
-		let options = {
-			params: {
-				ConditionExpression: this._makeItemExistsConditions(index)
-			}
-		};
+		let options = {};
 		return this._makeChain(index, this._clausesWithFilters, clauses.index, options).remove(facets);
 	}
 
@@ -866,29 +854,18 @@ class Entity {
 		return {parameters, config};
 	}
 
-	_makeItemDoesntExistConditions(index) {
-		let hasSortKey = this.model.lookup.indexHasSortKeys[index];
-		let accessPattern = this.model.translations.indexes.fromIndexToAccessPattern[index];
+	_getPrimaryIndexFieldNames() {
+		let hasSortKey = this.model.lookup.indexHasSortKeys[TableIndex];
+		let accessPattern = this.model.translations.indexes.fromIndexToAccessPattern[TableIndex];
 		let pkField = this.model.indexes[accessPattern].pk.field;
-		let filter = [`attribute_not_exists(${pkField})`];
+		let skField;
 		if (hasSortKey) {
-			let skField = this.model.indexes[accessPattern].sk.field;
-			filter.push(`attribute_not_exists(${skField})`);
+			skField = this.model.indexes[accessPattern].sk.field;
 		}
-		return filter.join(" AND ");
-	}
-
-	_makeItemExistsConditions(index) {
-		let hasSortKey = this.model.lookup.indexHasSortKeys[index];
-		let accessPattern = this.model.translations.indexes.fromIndexToAccessPattern[index];
-		let pkField = this.model.indexes[accessPattern].pk.field;
-
-		let filter = [`attribute_exists(${pkField})`];
-		if (hasSortKey) {
-			let skField = this.model.indexes[accessPattern].sk.field;
-			filter.push(`attribute_exists(${skField})`);
+		return {
+			pk: pkField,
+			sk: skField
 		}
-		return filter.join(" AND ");
 	}
 
 	_applyParameterExpressionTypes(params, filter) {
@@ -1885,54 +1862,105 @@ class Entity {
 		return utilities.formatKeyCasing(key, casing);
 	}
 
-	_findBestIndexKeyMatch(attributes) {
-		let candidates = this.model.facets.bySlot.map((val, i) => i);
+	_findBestIndexKeyMatch(attributes = {}) {
+		// an array of arrays, representing the order of pk and sk composites specified for each index, and then an
+		// array with each access pattern occupying the same array index.
 		let facets = this.model.facets.bySlot;
-		let match;
-		let keys = {};
-		for (let i = 0; i < facets.length; i++) {
-			let currentMatches = [];
-			let nextMatches = [];
-			for (let j = 0; j < candidates.length; j++) {
-				let slot = candidates[j];
-				if (!facets[i][slot]) {
-					continue;
-				}
-				let name = facets[i][slot].name;
-				let next = facets[i][slot].next;
-				let index = facets[i][slot].index;
-				let type = facets[i][slot].type;
-				let match = !!attributes[name];
-				let matchNext = !!attributes[next];
-				if (match) {
-					keys[index] = keys[index] || [];
-					keys[index].push({ name, type });
-					currentMatches.push(slot);
-					if (matchNext) {
-						nextMatches.push(slot);
-					}
+		// a flat array containing the match results of each access pattern, in the same array index they occur within
+		// bySlot above
+		let matches = [];
+		for (let f = 0; f < facets.length; f++) {
+			const slots = facets[f] || [];
+			for (let s = 0; s < slots.length; s++) {
+				const accessPatternSlot = slots[s];
+				matches[s] = matches[s] || {
+					index: accessPatternSlot.index,
+					allKeys: false,
+					hasSk: false,
+					count: 0,
+					done: false,
+					keys: []
 				}
-			}
-			if (currentMatches.length) {
-				if (nextMatches.length) {
-					candidates = [...nextMatches];
+				// already determined to be out of contention on prior iteration
+				const indexOutOfContention = matches[s].done;
+				// composite shorter than other indexes
+				const lacksAttributeAtSlot = !accessPatternSlot;
+				// attribute at this slot is not in the object provided
+				const attributeNotProvided = accessPatternSlot && attributes[accessPatternSlot.name] === undefined;
+				// if the next attribute is a sort key then all partition keys were provided
+				const nextAttributeIsSortKey = accessPatternSlot && accessPatternSlot.next && facets[f+1][s].type === "sk";
+				// if no keys are left then all attribute requirements were met (remember indexes don't require a sort key)
+				const hasAllKeys = accessPatternSlot && !accessPatternSlot.next;
+
+				// no sense iterating on items we know to be "done"
+				if (indexOutOfContention || lacksAttributeAtSlot || attributeNotProvided) {
+					matches[s].done = true;
 					continue;
-				} else {
-					match = facets[i][currentMatches[0]].index;
-					break;
 				}
-			} else if (i === 0) {
-				break;
-			} else {
-				match = (candidates[0] !== undefined && facets[i][candidates[0]].index) || TableIndex;
-				break;
+
+				// if the next attribute is a sort key (and you reached this line) then you have fulfilled all the
+				// partition key requirements for this index
+				if (nextAttributeIsSortKey) {
+					matches[s].hasSk = true;
+				// if you reached this step and there are no more attributes, then you fulfilled the index
+				} else if (hasAllKeys) {
+					matches[s].allKeys = true;
+				}
+
+				// number of successfully fulfilled attributes plays into the ranking heuristic
+				matches[s].count++;
+
+				// note the names/types of fulfilled attributes
+				matches[s].keys.push({
+					name: accessPatternSlot.name,
+					type: accessPatternSlot.type
+				});
 			}
 		}
-		return {
-			keys: keys[match] || [],
-			index: match || TableIndex,
-			shouldScan: match === undefined
-		};
+		// the highest count of matched attributes among all access patterns
+		let max = 0;
+		matches = matches
+			// remove incomplete indexes
+			.filter(match => match.hasSk || match.allKeys)
+			// calculate max attribute match
+			.map(match => {
+				max = Math.max(max, match.count);
+				return match;
+			});
+
+		// matched contains the ranked attributes. The closer an element is to zero the "higher" the rank.
+		const matched = [];
+		for (let m = 0; m < matches.length; m++) {
+			const match = matches[m];
+			// a finished primary index is most ideal (could be a get)
+			const primaryIndexIsFinished = match.index === "" && match.allKeys;
+			// if there is a tie for matched index attributes, primary index should win
+			const primaryIndexIsMostMatched = match.index === "" && match.count === max;
+			// composite attributes are complete
+			const indexRequirementsFulfilled = match.allKeys;
+			// having the most matches is important
+			const hasTheMostAttributeMatches = match.count === max;
+			if (primaryIndexIsFinished) {
+				matched[0] = match;
+			} else if (primaryIndexIsMostMatched) {
+				matched[1] = match;
+			} else if (indexRequirementsFulfilled) {
+				matched[2] = match;
+			} else if (hasTheMostAttributeMatches) {
+				matched[3] = match;
+			}
+		}
+		// find the first non-undefined element (best ranked) -- if possible
+		const match = matched.find(value => !!value);
+		let keys = [];
+		let index = "";
+		let shouldScan = true;
+		if (match) {
+			keys = match.keys;
+			index = match.index;
+			shouldScan = false;
+		}
+		return { keys, index, shouldScan };
 	}
 
 	/* istanbul ignore next */
@@ -1969,7 +1997,7 @@ class Entity {
 				type = "name";
 			} else if (char === "}" && type === "name") {
 				if (current.name.match(/^\s*$/)) {
-					throw new e.ElectroError(e.ErrorCodes.InvalidKeyCompositeAttributeTemplate, `Invalid key composite attribute template. Empty expression "\${${current.name}" provided. Expected attribute name.`);
+					throw new e.ElectroError(e.ErrorCodes.InvalidKeyCompositeAttributeTemplate, `Invalid key composite attribute template. Empty expression "\${${current.name}}" provided. Expected attribute name.`);
 				}
 				attributes.push({name: current.name, label: current.label});
 				current.name = "";

package/src/errors.js

@@ -207,28 +207,112 @@ const ErrorCodes = {
   },
 };
 
+function makeMessage(message, section) {
+  return `${message} - For more detail on this error reference: ${getHelpLink(section)}`
+}
+
 class ElectroError extends Error {
-  constructor(err, message) {
+  constructor(code, message) {
     super(message);
     let detail = ErrorCodes.UnknownError;
-    if (err && err.sym === ErrorCode) {
-      detail = err
+    if (code && code.sym === ErrorCode) {
+      detail = code
     }
-    this.message = `${message} - For more detail on this error reference: ${getHelpLink(detail.section)}`;
-
+    this._message = message;
+    // this.message = `${message} - For more detail on this error reference: ${getHelpLink(detail.section)}`;
+    this.message = makeMessage(message, detail.section);
     if (Error.captureStackTrace) {
       Error.captureStackTrace(this, ElectroError);
     }
 
     this.name = 'ElectroError';
-    this.ref = err;
+    this.ref = code;
     this.code = detail.code;
-    this.date = new Date();
+    this.date = Date.now();
     this.isElectroError = true;
   }
 }
 
+class ElectroValidationError extends ElectroError {
+  constructor(errors = []) {
+    const fields = [];
+    const messages = [];
+    for (let i = 0; i < errors.length; i++) {
+      const error = errors[i];
+      const message = error ? (error._message || error.message) : undefined;
+      messages.push(message);
+      if (error instanceof ElectroUserValidationError) {
+        fields.push({
+          field: error.field,
+          index: error.index,
+          reason: message,
+          cause: error.cause,
+          type: 'validation'
+        });
+      } else if (error instanceof ElectroAttributeValidationError) {
+        fields.push({
+          field: error.field,
+          index: error.index,
+          reason: message,
+          cause: error.cause || error, // error | undefined
+          type: 'validation'
+        });
+      } else if (message) {
+        fields.push({
+          field: '',
+          index: error.index,
+          reason: message,
+          cause: error !== undefined ? error.cause || error : undefined,
+          type: 'fatal'
+        });
+      }
+    }
+
+    const message = messages
+        .filter(message => typeof message === "string" && message.length)
+        .join(', ') || `Invalid value(s) provided`;
+
+    super(ErrorCodes.InvalidAttribute, message);
+    this.fields = fields;
+    this.name = "ElectroValidationError";
+  }
+}
+
+class ElectroUserValidationError extends ElectroError {
+  constructor(field, cause) {
+    let message;
+    let hasCause = false;
+    if (typeof cause === "string") {
+      message = cause;
+    } else if (cause !== undefined && typeof cause._message === "string" && cause._message.length) {
+      message = cause._message;
+      hasCause = true;
+    } else if (cause !== undefined && typeof cause.message === "string" && cause.message.length) {
+      message = cause.message;
+      hasCause = true;
+    } else {
+        message = "Invalid value provided";
+    }
+    super(ErrorCodes.InvalidAttribute, message);
+    this.field = field;
+    this.name = "ElectroUserValidationError";
+    if (hasCause) {
+      this.cause = cause;
+    }
+  }
+}
+
+class ElectroAttributeValidationError extends ElectroError {
+  constructor(field, reason) {
+    super(ErrorCodes.InvalidAttribute, reason);
+    this.field = field;
+  }
+}
+
 module.exports = {
+  ErrorCodes,
   ElectroError,
-  ErrorCodes
+  ElectroValidationError,
+  ElectroUserValidationError,
+  ElectroAttributeValidationError
 };

package/src/operations.js

@@ -1,6 +1,6 @@
 const {AttributeTypes, ItemOperations, AttributeProxySymbol, BuilderTypes} = require("./types");
 const e = require("./errors");
-const v = require("./util");
+const u = require("./util");
 
 const deleteOperations = {
     canNest: false,
@@ -21,6 +21,13 @@ const deleteOperations = {
 };
 
 const UpdateOperations = {
+    ifNotExists: {
+        template: function if_not_exists(options, attr, path, value) {
+            const operation = ItemOperations.set;
+            const expression = `${path} = if_not_exists(${path}, ${value})`;
+            return {operation, expression};
+        }
+    },
     name: {
         canNest: true,
         template: function name(options, attr, path) {
@@ -325,7 +332,7 @@ class AttributeOperationProxy {
     fromObject(operation, record) {
         for (let path of Object.keys(record)) {
             const value = record[path];
-            const parts = v.parseJSONPath(path);
+            const parts = u.parseJSONPath(path);
             let attribute = this.attributes;
             for (let part of parts) {
                 attribute = attribute[part];
@@ -342,7 +349,7 @@ class AttributeOperationProxy {
 
     fromArray(operation, paths) {
         for (let path of paths) {
-            const parts = v.parseJSONPath(path);
+            const parts = u.parseJSONPath(path);
             let attribute = this.attributes;
             for (let part of parts) {
                 attribute = attribute[part];
@@ -453,4 +460,9 @@ class AttributeOperationProxy {
     }
 }
 
-module.exports = {UpdateOperations, FilterOperations, ExpressionState, AttributeOperationProxy};
+const FilterOperationNames = Object.keys(FilterOperations).reduce((ops, name) => {
+    ops[name] = name;
+    return ops;
+}, {});
+
+module.exports = {UpdateOperations, FilterOperations, FilterOperationNames, ExpressionState, AttributeOperationProxy};

package/src/schema.js

@@ -289,7 +289,7 @@ class Attribute {
 
 	_makeCast(name, cast) {
 		if (cast !== undefined && !CastTypes.includes(cast)) {
-			throw new e.ElectroError(e.ErrorCodes.InvalidAttributeDefinition, `Invalid "cast" property for attribute: "${name}". Acceptable types include ${CastTypes.join(", ",)}`,
+			throw new e.ElectroError(e.ErrorCodes.InvalidAttributeDefinition, `Invalid "cast" property for attribute: "${name}". Acceptable types include ${CastTypes.join(", ")}`,
 		);
 		} else if (cast === AttributeTypes.string) {
 			return (val) => {
@@ -327,20 +327,34 @@ class Attribute {
 	_makeValidate(definition) {
 		if (typeof definition === "function") {
 			return (val) => {
-				let reason = definition(val);
-				return [!reason, reason || ""];
+				try {
+					let reason = definition(val);
+					const isValid = !reason;
+					if (isValid) {
+						return [isValid, []];
+					} else if (typeof reason === "boolean") {
+						return [isValid, [new e.ElectroUserValidationError(this.path, "Invalid value provided")]];
+					} else {
+						return [isValid, [new e.ElectroUserValidationError(this.path, reason)]];
+					}
+				} catch(err) {
+					return [false, [new e.ElectroUserValidationError(this.path, err)]];
+				}
 			};
 		} else if (definition instanceof RegExp) {
 			return (val) => {
 				if (val === undefined) {
-					return [true, ""];
+					return [true, []];
 				}
 				let isValid = definition.test(val);
-				let reason = isValid ? "" : `Invalid value for attribute "${this.path}": Failed model defined regex`;
+				let reason = [];
+				if (!isValid) {
+					reason.push(new e.ElectroUserValidationError(this.path, `Invalid value for attribute "${this.path}": Failed model defined regex`));
+				}
 				return [isValid, reason];
 			};
 		} else {
-			return (val) => [true, ""];
+			return () => [true, []];
 		}
 	}
 
@@ -385,15 +399,19 @@ class Attribute {
 
 	_isType(value) {
 		if (value === undefined) {
-			return [!this.required, this.required ? `Invalid value type at entity path: "${this.path}". Value is required.` : ""];
+			let reason = [];
+			if (this.required) {
+				reason.push(new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Value is required.`));
+			}
+			return [!this.required, reason];
 		}
 		let isTyped = false;
-		let reason = "";
+		let reason = [];
 		switch (this.type) {
 			case AttributeTypes.enum:
 				isTyped = this.enumArray.includes(value);
 				if (!isTyped) {
-					reason = `Invalid value type at entity path: "${this.path}". Value not found in set of acceptable values: ${u.commaSeparatedString(this.enumArray)}`;
+					reason.push(new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Value not found in set of acceptable values: ${u.commaSeparatedString(this.enumArray)}`));
 				}
 				break;
 			case AttributeTypes.any:
@@ -405,7 +423,7 @@ class Attribute {
 			default:
 				isTyped = typeof value === this.type;
 				if (!isTyped) {
-					reason = `Invalid value type at entity path: "${this.path}". Received value of type "${typeof value}", expected value of type "${this.type}"`;
+					reason.push(new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Received value of type "${typeof value}", expected value of type "${this.type}"`));
 				}
 				break;
 		}
@@ -414,12 +432,12 @@ class Attribute {
 
 	isValid(value) {
 		try {
-			let [isTyped, typeError] = this._isType(value);
-			let [isValid, validationError] = this.validate(value);
-			let reason = [typeError, validationError].filter(Boolean).join(", ");
-			return [isTyped && isValid, reason];
+			let [isTyped, typeErrorReason] = this._isType(value);
+			let [isValid, validationError] = isTyped ? this.validate(value) : [false, []];
+			let errors = [...typeErrorReason, ...validationError].filter(value => value !== undefined);
+			return [isTyped && isValid, errors];
 		} catch (err) {
-			return [false, err.message];
+			return [false, [err]];
 		}
 	}
 
@@ -433,10 +451,9 @@ class Attribute {
 
 	getValidate(value) {
 		value = this.val(value);
-		let [isValid, validationError] = this.isValid(value);
+		let [isValid, validationErrors] = this.isValid(value);
 		if (!isValid) {
-			// todo: #electroerror
-			throw new Error(validationError);
+			throw new e.ElectroValidationError(validationErrors);
 		}
 		return value;
 	}
@@ -509,13 +526,17 @@ class MapAttribute extends Attribute {
 
 	_isType(value) {
 		if (value === undefined) {
-			return [!this.required, this.required ? `Invalid value type at entity path: "${this.path}". Value is required.` : ""];
+			let reason = [];
+			if (this.required) {
+				reason.push(new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Value is required.`));
+			}
+			return [!this.required, reason];
 		}
 		const valueType = getValueType(value);
 		if (valueType !== ValueTypes.object) {
-			return [false, `Invalid value type at entity path "${this.path}. Received value of type "${valueType}", expected value of type "object"`];
+			return [false, [new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path "${this.path}. Received value of type "${valueType}", expected value of type "object"`)]];
 		}
-		let reason = "";
+		let reason = [];
 		const [childrenAreValid, childErrors] = this._validateChildren(value);
 		if (!childrenAreValid) {
 			reason = childErrors;
@@ -526,24 +547,24 @@ class MapAttribute extends Attribute {
 	_validateChildren(value) {
 		const valueType = getValueType(value);
 		const attributes = this.properties.attributes;
-		const errors = [];
+		let errors = [];
 		if (valueType === ValueTypes.object) {
 			for (const child of Object.keys(attributes)) {
-				const [isValid, errorMessages] = attributes[child].isValid(value === undefined ? value : value[child])
+				const [isValid, errorValues] = attributes[child].isValid(value === undefined ? value : value[child])
 				if (!isValid) {
-					errors.push(errorMessages);
+					errors = [...errors, ...errorValues]
 				}
 			}
 		} else if (valueType !== ValueTypes.object) {
 			errors.push(
-				`Invalid value type at entity path: "${this.path}". Expected value to be an object to fulfill attribute type "${this.type}"`
+				new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Expected value to be an object to fulfill attribute type "${this.type}"`)
 			);
 		} else if (this.properties.hasRequiredAttributes) {
 			errors.push(
-				`Invalid value type at entity path: "${this.path}". Map attribute requires at least the properties ${u.commaSeparatedString(Object.keys(attributes))}`
+				new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Map attribute requires at least the properties ${u.commaSeparatedString(Object.keys(attributes))}`)
 			);
 		}
-		return [errors.length === 0, errors.filter(Boolean).join(", ")];
+		return [errors.length === 0, errors];
 	}
 
 	val(value) {
@@ -560,7 +581,7 @@ class MapAttribute extends Attribute {
 		} else if (value && valueType !== "object" && Object.keys(value).length === 0) {
 			return getValue(value);
 		} else if (valueType !== "object") {
-			throw new Error(`Invalid value type at entity path: "${this.path}". Expected value to be an object to fulfill attribute type "${this.type}"`);
+			throw new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Expected value to be an object to fulfill attribute type "${this.type}"`);
 		}
 
 		const data = {};
@@ -643,24 +664,29 @@ class ListAttribute extends Attribute {
 	}
 
 	_validateArrayValue(value) {
+		const reason = [];
 		const valueType = getValueType(value);
 		if (value !== undefined && valueType !== ValueTypes.array) {
-			return [false, `Invalid value type at entity path "${this.path}. Received value of type "${valueType}", expected value of type "array"`];
+			return [false, [new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path "${this.path}. Received value of type "${valueType}", expected value of type "array"`)]];
 		} else {
-			return [true, ""];
+			return [true, []];
 		}
 	}
 
 	_isType(value) {
 		if (value === undefined) {
-			return [!this.required, this.required ? `Invalid value type at entity path: "${this.path}". Value is required.` : ""];
+			let reason = [];
+			if (this.required) {
+				reason.push(new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Value is required.`));
+			}
+			return [!this.required, reason];
 		}
 
 		const [isValidArray, errors] = this._validateArrayValue(value);
 		if (!isValidArray) {
 			return [isValidArray, errors];
 		}
-		let reason = "";
+		let reason = [];
 		const [childrenAreValid, childErrors] = this._validateChildren(value);
 		if (!childrenAreValid) {
 			reason = childErrors;
@@ -673,17 +699,22 @@ class ListAttribute extends Attribute {
 		const errors = [];
 		if (valueType === ValueTypes.array) {
 			for (const i in value) {
-				const [isValid, errorMessages] = this.items.isValid(value[i]);
+				const [isValid, errorValues] = this.items.isValid(value[i]);
 				if (!isValid) {
-					errors.push(errorMessages + ` at index "${i}"`);
+					for (const err of errorValues) {
+						if (err instanceof e.ElectroAttributeValidationError || err instanceof e.ElectroUserValidationError) {
+							err.index = parseInt(i);
+						}
+						errors.push(err);
+					}
 				}
 			}
 		} else {
 			errors.push(
-				`Invalid value type at entity path: "${this.path}". Expected value to be an Array to fulfill attribute type "${this.type}"`
+				new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Expected value to be an Array to fulfill attribute type "${this.type}"`)
 			);
 		}
-		return [errors.length === 0, errors.filter(Boolean).join(", ")];
+		return [errors.length === 0, errors];
 	}
 
 	val(value) {
@@ -700,7 +731,7 @@ class ListAttribute extends Attribute {
 		} else if (Array.isArray(value) && value.length === 0) {
 			return value;
 		} else if (!Array.isArray(value)) {
-			throw new Error(`Invalid value type at entity path "${this.path}. Received value of type "${getValueType(value)}", expected value of type "array"`);
+			throw new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path "${this.path}. Received value of type "${getValueType(value)}", expected value of type "array"`);
 		}
 
 		const data = [];
@@ -731,6 +762,20 @@ class SetAttribute extends Attribute {
 		this.items = items;
 		this.get = this._makeGet(definition.get, items);
 		this.set = this._makeSet(definition.set, items);
+		this.validate = this._makeSetValidate(definition);
+	}
+
+	_makeSetValidate(definition) {
+		const validate = this._makeValidate(definition.validate);
+		return (value) => {
+			if (Array.isArray(value)) {
+				return validate([...value]);
+			} else if (value && value.wrapperName === 'Set') {
+				return validate([...value.values])
+			} else {
+				return validate(value);
+			}
+		}
 	}
 
 	fromDDBSet(value) {
@@ -768,7 +813,7 @@ class SetAttribute extends Attribute {
 				return this._createDDBSet(value);
 			}
 			default:
-				throw new Error(`Invalid attribute value supplied to "set" attribute "${this.path}". Received value of type "${valueType}". Set values must be supplied as either Arrays, native JavaScript Set objects, DocumentClient Set objects, strings, or numbers.`)
+				throw new e.ElectroAttributeValidationError(this.path, `Invalid attribute value supplied to "set" attribute "${this.path}". Received value of type "${valueType}". Set values must be supplied as either Arrays, native JavaScript Set objects, DocumentClient Set objects, strings, or numbers.`)
 		}
 
 	}
@@ -795,7 +840,9 @@ class SetAttribute extends Attribute {
 		this._checkGetSet(set, "set");
 		const setter = set || ((attr) => attr);
 		return (values, siblings) => {
-			const results = setter(values, siblings);
+			const results = values && values.wrapperName === 'Set'
+				? setter(values.values, siblings)
+				: setter(values, siblings)
 			if (results !== undefined) {
 				return this.toDDBSet(results);
 			}
@@ -804,10 +851,14 @@ class SetAttribute extends Attribute {
 
 	_isType(value) {
 		if (value === undefined) {
-			return [!this.required, this.required ? `Invalid value type at entity path: "${this.path}". Value is required.` : ""];
+			const reason = [];
+			if (this.required) {
+				reason.push(new e.ElectroAttributeValidationError(this.path, `Invalid value type at entity path: "${this.path}". Value is required.`));
+			}
+			return [!this.required, reason];
 		}
 
-		let reason = "";
+		let reason = [];
 		const [childrenAreValid, childErrors] = this._validateChildren(value);
 		if (!childrenAreValid) {
 			reason = childErrors;
@@ -817,7 +868,7 @@ class SetAttribute extends Attribute {
 
 	_validateChildren(value) {
 		const valueType = getValueType(value);
-		const errors = [];
+		let errors = [];
 		let arr = [];
 		if (valueType === ValueTypes.array) {
 			arr = value;
@@ -827,16 +878,16 @@ class SetAttribute extends Attribute {
 			arr = value.values;
 		} else {
 			errors.push(
-				`Invalid value type at attribute path: "${this.path}". Expected value to be an Expected value to be an Array, native JavaScript Set objects, or DocumentClient Set objects to fulfill attribute type "${this.type}"`
+				new e.ElectroAttributeValidationError(this.path, `Invalid value type at attribute path: "${this.path}". Expected value to be an Expected value to be an Array, native JavaScript Set objects, or DocumentClient Set objects to fulfill attribute type "${this.type}"`)
 			)
 		}
 		for (const item of arr) {
-			const [isValid, errorMessage] = this.items.isValid(item);
+			const [isValid, errorValues] = this.items.isValid(item);
 			if (!isValid) {
-				errors.push(errorMessage);
+				errors = [...errors, ...errorValues];
 			}
 		}
-		return [errors.length === 0, errors.filter(Boolean).join(", ")];
+		return [errors.length === 0, errors];
 	}
 
 	val(value) {
@@ -1232,11 +1283,11 @@ class Schema {
 		for (const path of paths) {
 			const attribute = this.traverser.getPath(path);
 			if (!attribute) {
-				throw new Error(`Attribute "${path}" does not exist on model.`);
+				throw new e.ElectroAttributeValidationError(path, `Attribute "${path}" does not exist on model.`);
 			} else if (attribute.readOnly) {
-				throw new Error(`Attribute "${attribute.path}" is Read-Only and cannot be removed`);
+				throw new e.ElectroAttributeValidationError(attribute.path, `Attribute "${attribute.path}" is Read-Only and cannot be removed`);
 			} else if (attribute.required) {
-				throw new Error(`Attribute "${attribute.path}" is Required and cannot be removed`);
+				throw new e.ElectroAttributeValidationError(attribute.path, `Attribute "${attribute.path}" is Required and cannot be removed`);
 			}
 		}
 		return paths;
@@ -1251,7 +1302,7 @@ class Schema {
 			}
 			if (attribute.readOnly) {
 				// todo: #electroerror
-				throw new Error(`Attribute "${attribute.path}" is Read-Only and cannot be updated`);
+				throw new e.ElectroAttributeValidationError(attribute.path, `Attribute "${attribute.path}" is Read-Only and cannot be updated`);
 			} else {
 				record[path] = attribute.getValidate(value);
 			}

package/src/where.js

@@ -1,5 +1,5 @@
 const {MethodTypes, ExpressionTypes, BuilderTypes} = require("./types");
-const {AttributeOperationProxy, ExpressionState} = require("./operations");
+const {AttributeOperationProxy, ExpressionState, FilterOperations} = require("./operations");
 const e = require("./errors");
 
 class FilterExpression extends ExpressionState {
@@ -43,6 +43,22 @@ class FilterExpression extends ExpressionState {
 		this.expression = expression;
 	}
 
+	// applies operations without verifying them against known attributes. Used internally for key conditions.
+	unsafeSet(operation, name, ...values) {
+		const {template} = FilterOperations[operation] || {};
+		if (template === undefined) {
+			throw new Error(`Invalid operation: "${operation}". Please report`);
+		}
+		const names = this.setName({}, name, name);
+		if (values.length) {
+			for (const value of values) {
+				this.setValue(name, value);
+			}
+		}
+		const condition = template({}, name.expression, names.prop, ...values);
+		this.add(condition);
+	}
+
 	build() {
 		return this.expression;
 	}