@kensio/dynamodb-single-table 1.13.1

package/.claude-plugin/plugin.json

@@ -0,0 +1,14 @@
+{
+  "$schema": "https://anthropic.com/claude-code/plugin.schema.json",
+  "name": "dynamodb-single-table",
+  "version": "1.13.1",
+  "description": "Data modelling for Amazon DynamoDB, starting from one table.",
+  "author": {
+    "name": "Kensio Software",
+    "email": "hugh@kensiosoftware.co.uk"
+  },
+  "homepage": "https://kensio.ai",
+  "repository": "https://github.com/KensioSoftware/kensio.ai",
+  "license": "Apache-2.0",
+  "keywords": ["aws", "dynamodb", "nosql", "data-modelling", "single-table"]
+}

package/README.md

@@ -0,0 +1,123 @@
+# @kensio/dynamodb-single-table
+
+An agent skill for modelling data in Amazon DynamoDB, where the default is one table per service
+holding every entity type.
+
+Nearly all of the data modelling advice an LLM has read is about SQL, and it carries over badly. The
+result is a table per entity, a join in the application layer, and a `Scan` wherever the keys fall
+short. This skill interrupts that reflex and gives the modelling technique that replaces it.
+
+It follows the AWS guidance, which says the same thing. "You should maintain as few tables as
+possible in a DynamoDB application." The sources are two AWS posts,
+[Creating a single-table design](https://aws.amazon.com/blogs/compute/creating-a-single-table-design-with-amazon-dynamodb/)
+and
+[Single-table vs multi-table design](https://aws.amazon.com/blogs/database/single-table-vs-multi-table-design-in-amazon-dynamodb/),
+together with the
+[NoSQL design](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-general-nosql-design.html),
+[data modeling](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/data-modeling-foundations.html),
+[partitioning](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.Partitions.html)
+and
+[best practices](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/best-practices.html)
+pages of the developer guide.
+
+The skill carries the links and tells the agent to fetch them when designing or reviewing a real
+schema. A summary goes stale, quotas move, and the worked examples hold detail no summary keeps.
+
+## Install
+
+Into any agent that reads `SKILL.md`:
+
+```bash
+npx @kensio/skills add dynamodb-single-table
+```
+
+That copies the skill directory into `.agents/skills/`, where Codex, Cursor, Copilot, Gemini CLI and
+the other implementations of the specification look for one. Pass `--agent claude` for
+`.claude/skills/`, `--agent copilot` for `.github/skills/`, and `--user` to install it for every
+project at once.
+
+Claude Code also takes it as a plugin:
+
+```bash
+claude plugin marketplace add KensioSoftware/kensio.ai
+claude plugin install dynamodb-single-table@kensio
+```
+
+Or pin it in a repository as a dependency:
+
+```bash
+npm install @kensio/dynamodb-single-table
+```
+
+Every skill is also published as a zip on each
+[release](https://github.com/KensioSoftware/kensio.ai/releases), for a machine with no npm reach.
+Unzip it into `.agents/skills/` and it is installed.
+
+## What it covers
+
+**Write the access patterns down first.** AWS states this as a rule. Do not start designing the
+schema until the questions it answers are known. Each entry records what the caller holds, what
+comes back, the ordering and the cardinality. Data size, data shape and data velocity are what
+decide the keys. A pattern that arrives later costs a secondary index or a backfill, and never a new
+table.
+
+**One table, every entity type.** Items sharing a partition key form an item collection, held
+together and sorted by sort key. That is the whole mechanism. An order and its lines come back from
+one `Query`, where the multi-table version pays two round trips. One query for two items under 4 KB
+costs 0.5 read units eventually consistent, and two queries for the same items cost 1. One table
+also means one set of alarms, one backup policy, one stream and one key to rotate. Name it after the
+service, never after an entity, because a table called `Users` has already lost the argument.
+
+**Generic key names and entity prefixes.** `PK`, `SK`, `GSI1PK` and `GSI1SK`, with every value
+prefixed by its entity type and every item carrying a `type` attribute. A key named `customerId` can
+only ever hold a customer.
+
+**Overload the secondary indexes, and make them sparse.** Keep the number of indexes small. Every
+GSI is a full copy of the attributes it projects, rewritten whenever one of them changes, and a
+design that adds an index per access pattern feels the write cost long before it reaches the AWS
+ceiling of 20. The same index attributes carry different meanings for different entity types. An
+index keyed on an attribute only some items hold contains only those items, so setting `GSI2PK` to
+`STATUS#OPEN` while an order is open builds an index of open orders that reads no closed ones.
+
+**Split items by write rate.** A write is charged on the whole item rounded up to the kilobyte, so a
+view counter sharing an item with 2 KB of video metadata pays for the metadata on every increment.
+Keep the counter on its own item in the same partition. This split cuts across entity boundaries. A
+table-per-entity layout has no way to express it.
+
+**Uniqueness is a second item.** There is no unique index. The rule is an item keyed on the unique
+value, written in the same `TransactWriteItems` as the entity, both conditional on the key being
+free.
+
+**Many-to-many is an adjacency list.** Entities are partition keys and a relationship is an item in
+the partition keyed on the id at the other end. One inverted index, a GSI whose partition key is the
+table's sort key, buys the reverse of every relationship in the table.
+
+**Query, never Scan, and never select with a filter.** A `FilterExpression` runs after the read and
+is charged on everything read. Filtering 10,000 items down to 3 costs 10,000 items of read capacity.
+A single 1 MB scan page of 4 KB items costs 128 eventually consistent read units in one burst, taken
+from one partition, which throttles everything else sharing it.
+
+**Know the reasons for a second table.** Whole-table settings that need to differ (backup,
+encryption, table class), stream pressure past two consumers per shard, analytics exports,
+high-volume time series data, a different owning service, or a framework that fights the design.
+Each produces a second table holding several entity types. None of them produces a table per entity.
+
+## The reference file
+
+[`reference/aws-guidance.md`](skills/dynamodb-single-table/reference/aws-guidance.md) holds the
+mechanics the rules rest on, loaded only when a decision turns on them. Throughput per partition
+(3,000 read units and 1,000 write units), the AWS table of good and bad partition keys, write
+sharding with random and calculated suffixes, sort key hierarchy and version history patterns, index
+projections and LSI fetches, the 10 GB item collection limit, the three routes past 400 KB, what a
+`Scan` costs, the materialized graph pattern, and AWS's own list of what single-table design costs
+you.
+
+## Related skills
+
+- [`yulin-aws-simulation`](https://github.com/KensioSoftware/kensio.ai/tree/main/plugins/yulin-aws-simulation)
+  runs the tests for each access pattern against a simulated DynamoDB, using the real CDK table
+  definition.
+
+Part of [kensio.ai](https://github.com/KensioSoftware/kensio.ai). Licensed under the Apache License
+2.0. See the [LICENSE](https://github.com/KensioSoftware/kensio.ai/blob/main/LICENSE) in the
+repository root.

package/package.json

@@ -0,0 +1,38 @@
+{
+  "name": "@kensio/dynamodb-single-table",
+  "version": "1.13.1",
+  "description": "Data modelling for Amazon DynamoDB, starting from one table.",
+  "keywords": [
+    "agent-skills",
+    "aws",
+    "claude",
+    "claude-code",
+    "claude-code-plugin",
+    "codex",
+    "copilot",
+    "cursor",
+    "data-modelling",
+    "dynamodb",
+    "kensio",
+    "nosql",
+    "single-table",
+    "skill",
+    "skill-md"
+  ],
+  "homepage": "https://kensio.ai",
+  "license": "Apache-2.0",
+  "author": "Kensio Software <hugh@kensiosoftware.co.uk>",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/KensioSoftware/kensio.ai.git",
+    "directory": "plugins/dynamodb-single-table"
+  },
+  "files": [
+    ".claude-plugin",
+    "skills",
+    "README.md"
+  ],
+  "publishConfig": {
+    "access": "public"
+  }
+}

package/skills/dynamodb-single-table/SKILL.md

@@ -0,0 +1,329 @@
+---
+name: dynamodb-single-table
+description: Model data in Amazon DynamoDB as one table by default, reading the current AWS guidance before committing to a schema, writing the access patterns down before any keys exist, holding every entity type in one table, overloading generic partition and sort keys across those types, overloading and sparsifying secondary indexes, and splitting items by write rate. Use when designing or reviewing a DynamoDB schema, when a CDK stack is about to gain a second table, when an entity needs a query it has no key for, when application code fetches from two tables to assemble one response, when a Scan or a filter expression appears, when a partition runs hot or a write throttles, and when asked how to model users, orders, events or tenants in DynamoDB.
+license: Apache-2.0
+metadata:
+  version: "1.13.1"
+---
+
+# Single-table design in DynamoDB
+
+DynamoDB is a key-value store with a query language shaped like an index lookup. Almost all of the
+data modelling advice in the training data is about SQL, and carrying it over produces a table per
+entity, a join in the application layer, and a `Scan` wherever the keys fall short. That is the
+reflex this skill exists to interrupt.
+
+The default is **one table per service**, holding every entity type, with keys derived from the
+queries the application makes. AWS puts it plainly in the NoSQL design best practices. "You should
+maintain as few tables as possible in a DynamoDB application." A second table needs a reason from
+[When a second table earns its place](#when-a-second-table-earns-its-place). "One entity, one table"
+is never one of them.
+
+## Read the current guidance before designing a schema
+
+This file is a summary, and a summary goes stale. Quotas change, the worked examples carry detail no
+summary keeps, and a schema outlives the session that produced it. **Fetch the sources below when
+designing or reviewing a real schema**, and treat them as the authority wherever they disagree with
+what follows.
+
+The two posts, both worth reading end to end for the worked example:
+
+- [Creating a single-table design with Amazon DynamoDB](https://aws.amazon.com/blogs/compute/creating-a-single-table-design-with-amazon-dynamodb/)
+- [Single-table vs multi-table design in Amazon DynamoDB](https://aws.amazon.com/blogs/database/single-table-vs-multi-table-design-in-amazon-dynamodb/)
+
+The developer guide, starting with these four:
+
+- [NoSQL design for DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-general-nosql-design.html),
+  which is where the "as few tables as possible" rule lives.
+- [Data modeling foundations](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/data-modeling-foundations.html),
+  which sets out AWS's own advantages and disadvantages for both foundations.
+- [Partitions and data distribution](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.Partitions.html),
+  the mechanism everything else follows from.
+- [Best practices for designing and architecting with DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/best-practices.html),
+  an index whose sub-pages cover partition keys, sort keys, secondary indexes, large items, time
+  series data, many-to-many relationships and querying.
+
+[references/aws-guidance.md](references/aws-guidance.md) collects the mechanics from those sub-pages
+in one place, including throughput per partition, write sharding, index projections, sort key
+patterns and the read cost of a `Scan`. Read it when the question turns on performance or cost. The
+questions about shape are answered here.
+
+[NoSQL Workbench](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.html)
+is the AWS tool for building and visualising a model before writing any code, and it is worth
+suggesting to a user who is modelling something substantial.
+
+## Write the access patterns down first
+
+AWS states the ordering as a rule. Do not start designing the schema until the questions it answers
+are known. The schema is a consequence of the queries.
+
+Three properties matter before the first attribute is named.
+
+- **Data size**, meaning how much is stored and how much comes back in one request.
+- **Data shape**, meaning the shape in the table matches the shape the query wants, with no
+  reshaping at read time.
+- **Data velocity**, meaning where the peak load lands, which decides how the keys distribute.
+
+Enumerate the patterns and write them where the reviewer of the pull request can see them (the CDK
+stack, an ADR, a comment above the key builders). Each entry records what the caller holds, what
+comes back, the ordering and the expected cardinality. For a small orders service:
+
+| Access pattern                    | Caller holds | Returns               | Order        |
+| --------------------------------- | ------------ | --------------------- | ------------ |
+| Get a customer profile            | customer id  | one item              |              |
+| Get an order with its lines       | order id     | one order, 1-50 lines | line sku     |
+| List a customer's orders          | customer id  | 0-500 orders          | newest first |
+| Find a customer by email          | email        | one item              |              |
+| List open orders across customers | nothing      | 0-2000 orders         | oldest first |
+
+A pattern that arrives later usually costs a new secondary index or a backfill. It never costs a new
+table.
+
+## One table, every entity type
+
+Items sharing a partition key form an **item collection**, stored together and sorted by sort key.
+That is the whole mechanism behind single-table design. Related items of different types land in one
+collection and one `Query` returns them.
+
+The arguments for it, in the order AWS makes them:
+
+- **Locality of reference.** Keeping related data together is the first general principle in the
+  best practices, ahead of every key-design detail. An order and its lines share a partition, and
+  one `Query` returns both. The multi-table version issues a `GetItem` for the order and a `Query`
+  for the lines, and pays two round trips to assemble one response.
+- **Reads cost less.** One query for two items totalling under 4 KB is 0.5 read units eventually
+  consistent. Two queries for the same two items cost 1 read unit, because each is billed at 0.5.
+  Latency follows the same shape, and two calls average worse than one.
+- **Traffic smooths out.** Aggregating several usage patterns onto one table produces a steadier
+  overall curve than any single pattern has on its own, in the way an index moves more smoothly than
+  a share in it. Provisioned mode reaches a higher utilisation as a result.
+- **Cost tracks the item.** A write is charged per kilobyte of the whole item, so a view counter
+  living on the same item as the video metadata charges for the metadata on every increment.
+  Splitting by write rate is a modelling decision the table boundary cannot make.
+- **Operational surface stays flat.** One table means one set of alarms, one backup policy, one
+  capacity mode, one stream, one set of IAM statements, and one customer managed key to rotate.
+- **It keeps the modelling honest.** A table per entity looks like a relational schema and invites
+  relational habits. One table has no shape to fall back on except the access patterns.
+
+Name the table after the service (`orders-service-data`), never after an entity. A table called
+`Users` has already lost the argument.
+
+Symptoms of the relational reflex, all of which mean the keys are wrong:
+
+- Two or more `GetItem` calls in sequence to build one response.
+- A `Scan` with a `FilterExpression` doing the selection.
+- A secondary index added for each new query.
+- An `orderId` attribute on the customer item, used as a foreign key by the application.
+
+## Generic key names and entity prefixes
+
+Partition and sort key attributes are named `PK` and `SK`, and secondary index keys `GSI1PK`,
+`GSI1SK` and so on. A key named `customerId` can only ever hold a customer.
+
+Every value carries a prefix naming its entity type, and every item carries a `type` attribute for
+the deserialiser, for stream consumers and for exports.
+
+```
+PK               SK                    type       Attributes
+CUSTOMER#c-42    #PROFILE              customer   email, name, createdAt
+ORDER#o-981      #ORDER                order      customerId, status, total, placedAt
+ORDER#o-981      LINE#sku-0007         orderLine  qty, price
+ORDER#o-981      LINE#sku-0031         orderLine  qty, price
+```
+
+`Query` on `PK = "ORDER#o-981"` returns the order and every line in one request, in sort key order.
+`#ORDER` comes back first because `#` sorts below the letters. A `#` prefix is the usual trick for
+pinning a parent item to the top of its item collection.
+
+Sort keys are byte-ordered strings, and hierarchy in them is free. AWS gives
+`[country]#[region]#[state]#[county]#[city]#[neighborhood]` as the shape, queryable at every level
+with `begins_with` and `between`. A timestamp in ISO 8601 sorts chronologically without parsing. Pad
+numbers so `sku-10` sorts after `sku-9`.
+
+Choosing the partition key is also a throughput decision. A key with many distinct values used
+evenly (a customer id) distributes well, and one with few values (a status code) or one that
+concentrates writes on the current period (a date rounded to the day) does not. Every partition
+serves 3,000 read units and 1,000 write units per second.
+[references/aws-guidance.md](references/aws-guidance.md) covers the arithmetic, the AWS table of
+good and bad keys, and write sharding for the cases that need it.
+
+## Overload the secondary indexes
+
+Keep the number of indexes small. Every global secondary index is a full copy of the attributes it
+projects, rewritten whenever a projected attribute changes, and it consumes its own write capacity.
+AWS states the rule as "keep the number of indexes to a minimum" and warns that a seldom-used index
+costs storage and I/O without buying performance. The default quota caps a table at 20 of them, and
+a design that adds one index per access pattern will feel the write cost long before it reaches that
+ceiling.
+
+Index overloading is what keeps the count down. The same index attributes carry different meanings
+for different entity types:
+
+| Item      | GSI1PK                  | GSI1SK                     | Serves                   |
+| --------- | ----------------------- | -------------------------- | ------------------------ |
+| customer  | `EMAIL#ada@example.com` | `EMAIL`                    | find a customer by email |
+| order     | `CUSTOMER#c-42`         | `PLACED#2026-03-01T09:14Z` | a customer's orders      |
+| orderLine | (absent)                | (absent)                   |                          |
+
+Two access patterns, one index, and the line items stay out of it because they carry no `GSI1PK`.
+That last part is the **sparse index**. An index keyed on an attribute only some items hold contains
+only those items. Set `GSI2PK` to `STATUS#OPEN` while an order is open and remove the attribute when
+it closes. The index then holds the open orders and nothing else, and the query for them reads no
+closed orders at all.
+
+Prefer a GSI to an LSI. An LSI has to exist when the table is created, cannot be deleted afterwards,
+shares the table's throughput, and caps every item collection at 10 GB. A GSI can be added later and
+carries its own capacity. The case for an LSI is a strongly consistent read on an alternate sort
+key, which a GSI cannot give (GSI reads are eventually consistent, always).
+
+Decide the projection on every index. `ALL` removes fetches and roughly doubles storage and write
+cost. See [references/aws-guidance.md](references/aws-guidance.md) for the trade.
+
+## Split items by write rate
+
+The write cost of an item is its whole size rounded up to the next kilobyte. An attribute that
+changes constantly, sitting on an item that is mostly static, charges for the static part every
+time.
+
+```
+PK             SK          type       Attributes
+VIDEO#v-3      #METADATA   video      title, description, tags, uploadedAt   (2 KB, rarely written)
+VIDEO#v-3      #VIEWS      viewCount  count                                  (1 WCU per increment)
+```
+
+Same partition, so one `Query` still returns both. The counter now costs one write unit, down from
+two. This split cuts across entity boundaries. A table-per-entity layout has no way to express it.
+
+The same move handles items that outgrow 400 KB. Break the item into chunks under one partition key
+and order them by sort key, which AWS calls vertical partitioning.
+
+## Many-to-many is an adjacency list
+
+Model both sides in one table. Top-level entities are partition keys, and a relationship is an item
+inside the partition whose sort key is the id of the thing on the other end.
+
+```
+PK               SK               type
+INVOICE#i-3      #INVOICE         invoice
+INVOICE#i-3      BILL#b-7         invoiceBill
+INVOICE#i-3      BILL#b-9         invoiceBill
+BILL#b-7         #BILL            bill
+```
+
+A `Query` on `INVOICE#i-3` gives every bill on the invoice. The other direction comes from an
+**inverted index**, a GSI whose partition key is the table's sort key, so a query on `BILL#b-7`
+returns every invoice carrying that bill. One extra index buys the reverse of every relationship in
+the table.
+
+Where the traversal is graph-shaped and needs several hops at low latency, Amazon Neptune is the
+tool AWS points to.
+
+## Denormalise what is stable, and write the copies together
+
+A join is replaced either by co-location or by duplication. Copy the customer name onto the order
+item when the order view needs it, and keep the copies consistent in one `TransactWriteItems` (up to
+100 items in one call). Where the fan-out is too wide for a transaction, repair from a DynamoDB
+stream.
+
+Duplicate attributes that rarely change. An attribute that changes often turns every update into a
+fan-out write across every copy, and at that point a second `Query` for the current value is
+cheaper.
+
+## Uniqueness is a second item
+
+There is no unique index. A uniqueness rule is an item whose key is the unique value, written in the
+same transaction as the entity, both writes conditional on the key being free:
+
+```typescript
+await documents.send(new TransactWriteCommand({
+  TransactItems: [
+    {
+      Put: {
+        TableName: table,
+        Item: { PK: `CUSTOMER#${id}`, SK: "#PROFILE", type: "customer", email },
+        ConditionExpression: "attribute_not_exists(PK)",
+      },
+    },
+    {
+      Put: {
+        TableName: table,
+        Item: { PK: `EMAIL#${email}`, SK: "#EMAIL", type: "emailClaim", customerId: id },
+        ConditionExpression: "attribute_not_exists(PK)",
+      },
+    },
+  ],
+}));
+```
+
+Either both land or neither does. The claim item is another entity type in the same table, and it
+needs releasing when the email changes.
+
+## Query, never Scan, and never select with a filter
+
+A `FilterExpression` is applied after the read and charged on everything read. Filtering 10,000
+items down to 3 costs 10,000 items of read capacity and returns pages that look half-empty. Filters
+are for trimming a result set the key condition has already narrowed, and never for choosing which
+items to fetch.
+
+A `Scan` in a request path is a modelling failure. It reads the whole table, slows as the table
+grows, and takes its capacity from one partition at a time, which throttles the requests that share
+that partition. A single 1 MB page of 4 KB items costs 128 eventually consistent read units in one
+burst.
+
+In a batch job over the whole table a `Scan` is legitimate. Set `Limit` to cap the page size and use
+parallel segments once the table passes about 20 GB.
+
+Watch the 1 MB page limit on `Query` too. Code that reads `Items` without following
+`LastEvaluatedKey` silently truncates.
+
+## When a second table earns its place
+
+Each of these is a real reason, and each produces a second table holding several entity types. None
+of them produces a table per entity.
+
+- **Whole-table settings that need to differ.** Backups and point-in-time recovery are per table,
+  and so are encryption keys and the table class. Mission-critical data mixed with disposable data
+  gets backed up as one unit. A multi-tenant application needing a key per tenant needs a table per
+  tenant or client-side encryption. Historical data mixed with operational data loses most of the
+  Infrequent Access saving.
+- **Stream pressure.** One stream carries every change to every entity type, and a shard supports
+  about two concurrent readers before throttling. Entities needing separate downstream pipelines
+  (orders into Step Functions, registrations into EventBridge) can exhaust that. Lambda event
+  filters keep the irrelevant records off the bill, and the Kinesis Client Library does not.
+- **Analytics exports.** An immutable event log and a mutable entity set want different export
+  strategies. Full exports suit mutable data and streaming suits the log.
+- **High-volume time series data.** AWS names this as an explicit exception, along with datasets
+  whose access patterns have nothing in common. A table per time period is the usual shape.
+- **A different owner.** Another service owning the data means a shared table couples two
+  deployments and two IAM policies. The table boundary follows the service boundary.
+- **A framework that fights it.** GraphQL resolvers map cleanly onto one entity per table, and
+  higher-level SDK mappers struggle when one response holds several classes. AWS lists both as
+  disadvantages of single-table design.
+
+Absent one of these, the second table is the SQL habit reasserting itself.
+
+## Keep key construction in one place
+
+Single-table keys are strings with structure, and structure spread across handlers as template
+literals drifts. One module owns building and parsing them, and nothing else concatenates a `#`.
+
+```typescript
+export const keys = {
+  customer: (id: string) => ({ PK: `CUSTOMER#${id}`, SK: "#PROFILE" }),
+  order: (orderId: string) => ({ PK: `ORDER#${orderId}`, SK: "#ORDER" }),
+  orderLine: (orderId: string, sku: string) => ({ PK: `ORDER#${orderId}`, SK: `LINE#${sku}` }),
+  ordersForCustomer: (id: string) => ({ GSI1PK: `CUSTOMER#${id}` }),
+};
+```
+
+A `Query` over one partition returns several entity types, so the item type is a discriminated union
+keyed on the `type` attribute. Parse into that union at the edge of the data access layer and let
+the rest of the code hold real types.
+
+## Testing the model
+
+Access patterns are testable, and each one deserves a test that seeds a few items and asserts what
+the query returns. The
+[`yulin-aws-simulation`](https://github.com/KensioSoftware/kensio.ai/tree/main/plugins/yulin-aws-simulation)
+skill covers running those tests against a simulated DynamoDB with the real CDK table definition,
+including key schema and index projections.

package/skills/dynamodb-single-table/references/aws-guidance.md

@@ -0,0 +1,183 @@
+# Mechanics behind the modelling rules
+
+The numbers and patterns the [SKILL.md](../SKILL.md) rules rest on, collected from the DynamoDB
+developer guide. Read this when the question turns on cost, throughput or size. Fetch the linked
+pages when a decision rests on an exact figure, because quotas move.
+
+## Partitions and throughput
+
+A table is stored in partitions, each backed by SSD and replicated across Availability Zones. AWS
+manages them and never exposes them directly. More partitions appear when provisioned throughput
+rises past what the current ones serve, and when an existing partition fills.
+
+DynamoDB hashes the partition key to choose the partition. Items sharing a partition key value form
+an **item collection**, held together and sorted by sort key, which is what makes a range query over
+one collection cheap. Where the table carries no local secondary index, DynamoDB splits an item
+collection across as many partitions as it needs, and there is no ceiling on the number of distinct
+sort key values under one partition key.
+
+**Every partition serves 3,000 read units and 1,000 write units per second.** One read unit is one
+strongly consistent read of an item up to 4 KB, or two eventually consistent reads. One write unit
+is one write of an item up to 1 KB. Item size multiplies this. A 20 KB item costs 5 read units per
+consistent read, which puts the ceiling at 600 reads per second against that one item.
+
+Source
+[Partitions and data distribution](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.Partitions.html)
+and
+[Best practices for partition keys](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-design.html).
+
+## Choosing a partition key that spreads
+
+Throughput efficiency rises with the ratio of partition key values accessed to partition key values
+that exist. The AWS comparison:
+
+| Partition key value                                             | Uniformity |
+| --------------------------------------------------------------- | ---------- |
+| User id, in an application with many users                      | Good       |
+| Status code, where few codes exist                              | Bad        |
+| Creation date rounded to a day, hour or minute                  | Bad        |
+| Device id, where devices are accessed at similar intervals      | Good       |
+| Device id, where one device is far more popular than the others | Bad        |
+
+The date case is the one that catches people. Every item created today lands on one partition key
+value and therefore one physical partition.
+
+A table small enough to fit in a single partition, allowing for growth, and whose throughput stays
+inside one partition's limits, will not throttle whatever the key looks like.
+
+Source
+[Designing partition keys to distribute your workload](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-uniform-load.html).
+
+## Write sharding
+
+Where the natural key concentrates writes, widen the key space by appending a suffix.
+
+**Random suffix.** Append a random number in a fixed range, giving `2026-07-09.1` through
+`2026-07-09.200`. Writes spread evenly. Reading one item back becomes impossible without knowing
+which suffix it took, and reading the whole day means one `Query` per suffix followed by a merge.
+
+**Calculated suffix.** Derive the suffix from an attribute the reader already holds, such as the sum
+of the UTF-8 code points of an order id modulo 200 plus 1. Writes spread the same way, and a
+`GetItem` for a known order still works because the suffix is recomputable. Reading the whole day
+still costs one `Query` per suffix.
+
+A GSI can be sharded the same way to make selective queries parallel.
+
+Source
+[Using write sharding](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-partition-key-sharding.html).
+
+## Sort key patterns
+
+**Hierarchy.** A composite sort key defines one-to-many relationships queryable at any level, using
+`begins_with`, `between`, `>` and `<`. The AWS example is
+`[country]#[region]#[state]#[county]#[city]#[neighborhood]`.
+
+**Version history.** Keep two copies of every item. One carries a `v0_` sort key prefix and holds
+the current version, and each revision is written under the next number up (`v1_`, `v2_` and so on)
+with its contents also copied over `v0_`. The current version is then a query on the `v0_` prefix,
+and the history is the rest of the partition.
+
+Source
+[Best practices for sort keys](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-sort-keys.html).
+
+## Secondary indexes
+
+A table gets 20 global secondary indexes (default quota) and 5 local secondary indexes. AWS says
+global indexes are usually the more useful of the two.
+
+**Keep the number to a minimum.** An index that is seldom queried adds storage and I/O cost and buys
+no performance.
+
+**Choose projections deliberately.** A smaller index costs less and outperforms the base table by
+more. Project the attributes the queries actually return. `ALL` removes every fetch and in most
+cases doubles storage and write cost. Where an index entry is under 1 KB the projection is free up
+to that point, because writes round up.
+
+**Avoid fetches on the read path.** Querying an LSI for an attribute it does not project makes
+DynamoDB read the whole item from the table, adding latency and I/O. Attributes queried occasionally
+have a habit of becoming attributes queried always.
+
+**Watch LSI item collections.** An item collection covers the table items and every LSI item sharing
+a partition key, and it cannot exceed 10 GB. Writes fail once it does. Pass
+`ReturnItemCollectionMetrics` on writes and alarm before the limit. An LSI cannot be deleted after
+creation, which makes the decision to add one permanent.
+
+Source
+[General guidelines for secondary indexes](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-indexes-general.html).
+
+## Large items
+
+The item size limit is 400 KB, and exceeding it fails the write with a `ValidationException`. Pass
+`ReturnConsumedCapacity` on writes and alarm on items approaching the limit.
+
+Three ways out, in the order worth trying:
+
+- **Vertical partitioning.** Break the item into several items under one partition key, ordered by
+  sort key. This is the single-table answer and keeps everything queryable.
+- **Compression.** GZIP or LZO into a `Binary` attribute. A compressed attribute cannot be filtered
+  or queried on.
+- **S3.** Store the payload as an object and the object key in the item, with the item's primary key
+  in the S3 object metadata pointing back. No transaction spans the two, so the application owns the
+  cleanup of orphaned objects.
+
+Source
+[Best practices for storing large items](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-use-s3-too.html).
+
+## What a Scan actually costs
+
+A `Scan` reads the whole table or index and then discards what the filter rejects. It slows as the
+table grows.
+
+A single 1 MB page of 4 KB items costs 128 eventually consistent read units, or 256 strongly
+consistent. That arrives as one spike, and it lands on one partition, because the items a scan reads
+sit next to each other. Requests sharing that partition throttle.
+
+Where a scan is needed:
+
+- Set `Limit` to shrink the page, which spreads the cost and leaves gaps for other traffic.
+- Use parallel segments once the table passes about 20 GB, starting at roughly one segment per 2 GB,
+  and only where the provisioned read capacity is not already busy.
+- Retry throttled requests with exponential backoff.
+
+Source
+[Best practices for querying and scanning](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-query-scan.html).
+
+## Adjacency lists and the materialized graph
+
+Top-level entities become partition keys, and each relationship becomes an item in the partition
+whose sort key is the id at the other end. Data duplication stays minimal and the forward query is a
+plain `Query`.
+
+The reverse direction comes from an **inverted index**, a global secondary index whose partition key
+is the table's sort key.
+
+The materialized graph pattern extends this. Edge items carry `Type` and `Target` attributes
+composed into a `TypeTarget` key, one overloaded GSI indexes a `Data` attribute holding dates,
+names, places and skills, and a second GSI on `TypeTarget` answers reverse lookups. Aggregations
+large enough to run hot (everyone born on one date, everyone with one skill) want sharding across
+logical partitions.
+
+Multi-hop traversal at millisecond latency is Amazon Neptune's job. AWS says so on the many-to-many
+page itself.
+
+Source
+[Best practices for many-to-many relationships](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-adjacency-graphs.html).
+
+## AWS's own trade-off lists
+
+Worth reading in full at
+[Data modeling foundations](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/data-modeling-foundations.html),
+because they are the most honest account of the cost of single-table design.
+
+What AWS counts against single-table design. The learning curve is steep because the design runs
+opposite to relational instinct, whole-table settings (backup, encryption, table class) apply to
+every entity at once, streams carry every change whether or not a consumer wants it, GraphQL is
+harder to implement, and higher-level SDK mappers struggle with one response holding several
+classes.
+
+What AWS counts for it. Data locality, fewer read units and fewer round trips, one set of
+permissions and alarms, one key to rotate, capacity averaged across entities, and traffic that
+smooths as patterns aggregate.
+
+AWS's summary of when multiple tables are the right answer is short. Where the access patterns never
+query several entities together, multiple tables are good and sufficient.