@eresearchqut/ddb-repository 1.5.8 → 1.13.5

package/README.md

@@ -8,139 +8,244 @@ A TypeScript library providing a generic repository pattern implementation for A
 ## Features
 
 - 🚀 Generic repository pattern for type-safe DynamoDB operations
-- 📦 Simple and intuitive API
-- 🔍 Support for common CRUD operations (Create, Read, Update, Delete)
-- 🎯 Batch operations support
+- 🔍 Rich query support with filter expressions and projections
+- 🎯 Batch get with automatic retry for unprocessed keys
+- 📄 Paginated queries with limit and sort order control
+- 🗂️ [JSON Pointer Repository](#jsonpointerrepository) for storing structured JSON documents
 - 🧪 Fully tested with Jest and Testcontainers
 - 💪 Written in TypeScript with full type safety
 - ⚡ Built on top of AWS SDK v3
 
 ## Installation
+
 ```sh
 npm install @eresearchqut/ddb-repository
 ```
+
 ## Prerequisites
 
-- Node.js
+- Node.js LTS
 - AWS credentials configured (for production use)
-- DynamoDB table with appropriate schema
+- DynamoDB table with the appropriate key schema
 
 ## Usage
 
 ### Basic Example
+
 ```typescript
-import { DynamoDbRepository } from 'ddb-repository';
+import { DynamoDbRepository, FilterOperator } from '@eresearchqut/ddb-repository';
 import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
 
-// Define your entity type
+interface UserKey {
+  id: string;
+}
+
 interface User {
-id: string;
-name: string;
-email: string;
-createdAt: string;
+  id: string;
+  name: string;
+  email?: string;
+  age?: number;
 }
 
-// Initialize DynamoDB client
 const client = new DynamoDBClient({ region: 'us-east-1' });
 
-// Create repository instance
-const userRepository = new DynamoDbRepository<User>(
-client,
-'users-table',
-'id' // partition key
-);
+const userRepository = new DynamoDbRepository<UserKey, User>({
+  client,
+  tableName: 'users',
+  hashKey: 'id',
+});
+
+// Write an item
+await userRepository.putItem({ id: '123' }, { id: '123', name: 'Alice', email: 'alice@example.com' });
 
-// Create a new user
-await userRepository.create({
-id: '123',
-name: 'John Doe',
-email: 'john@example.com',
-createdAt: new Date().toISOString()
+// Read an item
+const user = await userRepository.getItem({ id: '123' });
+
+// Update specific fields (SET name, REMOVE email)
+await userRepository.updateItem({ id: '123' }, { name: 'Alicia' }, ['email']);
+
+// Delete an item
+await userRepository.deleteItem({ id: '123' });
+```
+
+### Querying Items
+
+`getItems` performs a DynamoDB `Query` and returns all matching items. It paginates automatically and stops early once `limit` items are accumulated.
+
+```typescript
+// All items for a partition key
+const items = await userRepository.getItems({ id: 'user-123' });
+
+// With filter expressions
+const activeAdults = await userRepository.getItems({
+  id: 'user-123',
+  filterExpressions: [
+    { attribute: 'age', operator: FilterOperator.GREATER_THAN_OR_EQUALS, value: 18 },
+    { attribute: 'status', operator: FilterOperator.EQUALS, value: 'active' },
+  ],
 });
 
-// Find a user by ID
-const user = await userRepository.findById('123');
+// With projection (only return selected attributes)
+const names = await userRepository.getItems({
+  id: 'user-123',
+  projectedAttributes: ['id', 'name'],
+});
 
-// Update a user
-await userRepository.update('123', {
-email: 'newemail@example.com'
+// With limit and sort order
+const recent = await userRepository.getItems({
+  id: 'user-123',
+  limit: 10,
+  sortOrder: 'DESC',
 });
 
-// Delete a user
-await userRepository.delete('123');
+// Query a Global Secondary Index
+const byStatus = await userRepository.getItems({
+  status: 'active',
+  index: 'status-index',
+});
 ```
 
-## API Reference
+### Filter Operators
 
-### Constructor
-```
-typescript
-new DynamoDbRepository<T>(client: DynamoDBClient, tableName: string, partitionKey: string, sortKey?: string)
-```
-### Methods
+| Operator | DynamoDB Expression | Notes |
+|---|---|---|
+| `EQUALS` | `#attr = :val` | |
+| `NOT_EQUALS` | `#attr <> :val` | |
+| `GREATER_THAN` | `#attr > :val` | |
+| `GREATER_THAN_OR_EQUALS` | `#attr >= :val` | |
+| `LESS_THAN` | `#attr < :val` | |
+| `LESS_THAN_OR_EQUALS` | `#attr <= :val` | |
+| `IN` | `#attr IN (:val0, :val1, ...)` | Pass an `Array` as value |
+| `BETWEEN` | `#attr BETWEEN :val0 AND :val1` | Pass a 2-tuple as value |
+| `BEGINS_WITH` | `begins_with(#attr, :val)` | String prefix match |
+| `CONTAINS` | `contains(#attr, :val)` | Substring or set membership |
 
-- `create(item: T): Promise<T>` - Create a new item
-- `findById(id: string): Promise<T | null>` - Find item by partition key
-- `update(id: string, updates: Partial<T>): Promise<T>` - Update an existing item
-- `delete(id: string): Promise<void>` - Delete an item
-- `findAll(): Promise<T[]>` - Retrieve all items (use with caution on large tables)
-- `batchCreate(items: T[]): Promise<void>` - Create multiple items in batch
-- `query(options: QueryOptions): Promise<T[]>` - Query items with custom conditions
+All operators support `negate: true` to wrap the expression in `NOT (...)`.
 
-## Development
+### Batch Get
 
-### Setup
-```sh
-# Install dependencies
-npm install
+```typescript
+const keys = [{ id: '1' }, { id: '2' }, { id: '3' }];
+const items = await userRepository.batchGetItems(keys);
+```
 
-# Run linter
-npm run lint
+Automatically deduplicates keys, pages requests in chunks of 100, and retries any `UnprocessedKeys` with exponential back-off.
 
-# Fix linting issues automatically
-npm run lint:fix
+### Composite Key Tables
 
-# Run tests
-npm test
+```typescript
+interface OrderKey {
+  customerId: string;
+  orderId: string;
+}
 
-# Run tests in watch mode
-npm run test:watch
+interface Order {
+  customerId: string;
+  orderId: string;
+  total: number;
+}
 
-# Run tests with coverage report
-npm run test:coverage
+const orderRepository = new DynamoDbRepository<OrderKey, Order>({
+  client,
+  tableName: 'orders',
+  hashKey: 'customerId',
+  rangeKey: 'orderId',
+});
 
-# Build the project
-npm run build
+await orderRepository.putItem(
+  { customerId: 'c1', orderId: 'o1' },
+  { customerId: 'c1', orderId: 'o1', total: 99.99 },
+);
 ```
-### Testing
 
-The project uses Jest with Testcontainers for integration testing against a real DynamoDB instance:
-```sh
-npm test
+### Consumed Capacity Middleware
+
+```typescript
+import { consumedCapacityMiddleware } from '@eresearchqut/ddb-repository';
+
+client.middlewareStack.add(
+  consumedCapacityMiddleware({
+    onConsumedCapacity: async (detail) => {
+      console.log('Consumed capacity:', detail.ConsumedCapacity);
+    },
+  }),
+);
 ```
 
-### Run tests with coverage
+## API Reference
 
-Generate coverage report
+### `DynamoDbRepository<K, T>`
 
-```sh
-npm run test:coverage
+#### Constructor options
+
+| Option | Type | Required | Description |
+|---|---|---|---|
+| `client` | `DynamoDBClient` | ✅ | AWS SDK v3 DynamoDB client |
+| `tableName` | `string` | ✅ | DynamoDB table name |
+| `hashKey` | `string` | ✅ | Partition key attribute name |
+| `rangeKey` | `string` | | Sort key attribute name |
+| `returnConsumedCapacity` | `ReturnConsumedCapacity` | | Defaults to `TOTAL` |
+
+#### Methods
+
+| Method | Signature | Description |
+|---|---|---|
+| `getItem` | `(key: K) => Promise<T \| undefined>` | Read a single item by key |
+| `putItem` | `(key: K, record: T) => Promise<T>` | Write an item (create or replace) |
+| `updateItem` | `(key: K, updates: Partial<T>, remove?: string[]) => Promise<T \| undefined>` | Partial update with optional attribute removal |
+| `deleteItem` | `(key: K) => Promise<Partial<T> \| undefined>` | Delete an item |
+| `getItems` | `(query: Query) => Promise<T[] \| undefined>` | Query items (auto-paginates) |
+| `batchGetItems` | `(keys: K[], projectedQuery?: ProjectedQuery) => Promise<Array<T \| undefined>>` | Fetch multiple items by key |
+
+## JsonPointerRepository
+
+Stores JSON documents as individual per-pointer DynamoDB items using [RFC 6901 JSON Pointer](https://datatracker.ietf.org/doc/html/rfc6901) addressing.
+
+See [JSON_POINTER_REPOSITORY.md](./JSON_POINTER_REPOSITORY.md) for full documentation.
+
+### Quick Example
+
+```typescript
+import { JsonPointerRepository } from '@eresearchqut/ddb-repository';
+
+const docRepo = new JsonPointerRepository({
+  client,
+  tableName: 'documents',  // requires hash key "id", range key "pointer"
+});
+
+await docRepo.putDocument('doc-1', {
+  name: 'Alice',
+  address: { city: 'Melbourne' },
+  tags: ['admin', 'user'],
+});
+
+const doc = await docRepo.getDocument('doc-1');
+// { name: 'Alice', address: { city: 'Melbourne' }, tags: ['admin', 'user'] }
+
+const city = await docRepo.getAttribute('doc-1', '/address/city');
+// 'Melbourne'
+
+await docRepo.putAttribute('doc-1', '/address/postcode', '3000');
+await docRepo.deleteAttribute('doc-1', '/tags/1');
+await docRepo.deleteDocument('doc-1');
 ```
 
-Coverage reports are generated in the coverage/ directory:
-* coverage/lcov-report/index.html - Interactive HTML report
-* coverage/lcov.info - LCOV format for CI/CD integration
+## Development
 
-View HTML coverage report, open coverage/lcov-report/index.html
+```sh
+npm install        # install dependencies
+npm run lint       # lint
+npm run lint:fix   # lint with auto-fix
+npm test           # run integration tests (requires Docker)
+npm run test:coverage  # tests with coverage report
+npm run build      # compile to dist/
+```
 
 ## Configuration
 
 ### AWS Credentials
 
-Ensure your AWS credentials are configured via:
-- Environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`)
-- AWS credentials file (`~/.aws/credentials`)
-- IAM role (when running on AWS infrastructure)
+Configure via environment variables (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`), AWS credentials file (`~/.aws/credentials`), or IAM role.
 
 ### Required IAM Permissions
 
@@ -156,8 +261,7 @@ Ensure your AWS credentials are configured via:
         "dynamodb:UpdateItem",
         "dynamodb:DeleteItem",
         "dynamodb:Query",
-        "dynamodb:Scan",
-        "dynamodb:BatchWriteItem"
+        "dynamodb:BatchGetItem"
       ],
       "Resource": "arn:aws:dynamodb:*:*:table/your-table-name"
     }
@@ -167,37 +271,18 @@ Ensure your AWS credentials are configured via:
 
 ## Contributing
 
-Contributions are welcome! Please feel free to submit a Pull Request.
-
+Contributions are welcome! Please submit a Pull Request.
 
 ### Commit message convention
-Semantic release uses conventional commits. Your commit messages should follow this format:
-
-* feat: new feature → triggers minor version bump (1.x.0)
-* fix: bug fix → triggers patch version bump (1.0.x)
-* perf: performance improvement → triggers patch version bump
-* docs: documentation change → no release
-* chore: maintenance task → no release
-* BREAKING CHANGE: in footer → triggers major version bump (x.0.0)
-
-Example:
-
-> feat: add batch write support
->
-> Added support for batch write operations to improve performance
 
-Or with breaking change:
-
-> feat: change repository API
-> 
-> BREAKING CHANGE: The query method now returns a Promise instead of an Observable
+Semantic release uses [Conventional Commits](https://www.conventionalcommits.org/):
 
+- `feat:` → minor version bump
+- `fix:` / `perf:` → patch version bump
+- `docs:` / `chore:` → no release
+- `feat!:` or `BREAKING CHANGE:` footer → major version bump
 
 ## License
 
 MIT
 
-## Support
-
-For issues and questions, please open an issue on the GitHub repository.
-