graphddb 0.4.2 → 0.5.0

package/docs/cloudformation.md

@@ -0,0 +1,294 @@
+# GraphDDB — CloudFormation generation
+
+`graphddb generate cloudformation` emits an AWS CloudFormation template
+describing the **DynamoDB table(s)** a GraphDDB single-table design maps onto.
+GraphDDB is single-table by design: every entity that shares a `table` name
+contributes to the SAME `AWS::DynamoDB::Table`, and their Global Secondary
+Indexes are **unioned** by index name.
+
+The command consumes the same serializable manifest the Python bridge uses (it
+loads your entry model, builds the manifest from the shared metadata registry,
+and emits the template). Output is **deterministic** — the same model set always
+produces byte-identical output — and defaults to YAML (`--format json` for JSON).
+
+> This document describes the shipped `generate cloudformation` command. It is
+> the reference for the command's options, the resources it generates, the
+> derivation rules that map a model set onto a table, and its scope boundary.
+
+### Related specifications
+
+- [`spec.md`](./spec.md) — the core API: entities, structured keys / GSIs, the
+  physical key attributes (`PK` / `SK` / `<index>PK` / `<index>SK`).
+- [`design-patterns.md`](./design-patterns.md) — maintained access paths and the
+  `updateMode: 'stream'` signal that drives `--stream auto`.
+- [`python-bridge.md`](./python-bridge.md) — the other consumer of the same
+  manifest / SSoT.
+
+## Overview
+
+```bash
+graphddb generate cloudformation --entry models.ts --out table.yaml
+```
+
+- **`--entry`** loads the TypeScript (or pre-compiled `.js`/`.mjs`) module that
+  registers your entity models as a side effect of import. TypeScript modules are
+  transpiled on demand via `tsx` (an optional peer dependency — install it with
+  `npm i -D tsx`).
+- Without `--out`, the template is written verbatim to **stdout** (pipe-friendly).
+  With `--out`, it is written to that file and a JSON result
+  (`{"status":"ok","outDir":…,"files":[…]}`) is printed to stdout.
+
+## Options
+
+Every option below matches the shipped CLI contract (`cli-contract.yaml`) and the
+handler validation exactly.
+
+| Option | Alias | Type | Default | Notes / validation |
+|--------|-------|------|---------|--------------------|
+| `--entry` | `-e` | file | — (required) | Entry module that registers the entity models. Must exist / be readable; missing → exit `2`. |
+| `--out` | `-o` | file | stdout | Output file. Parent directories are created. Omitted → template to stdout. |
+| `--format` | `-f` | `yaml` \| `json` | `yaml` | Serialization format. Any other value → exit `2`. |
+| `--gsi-projection` | | `ALL` \| `KEYS_ONLY` | `ALL` | GSI `ProjectionType`. `ALL` projects every attribute; `KEYS_ONLY` projects only the key attributes. Other value → exit `2`. |
+| `--stream` | | `auto` \| `on` \| `off` | `auto` | DynamoDB Streams (`StreamSpecification` `NEW_AND_OLD_IMAGES`). `auto` emits a stream only when the model set uses stream-based maintenance (`updateMode: 'stream'`); `on` always emits; `off` never. When enabled, a `StreamArn` Output is added. Other value → exit `2`. |
+| `--billing-mode` | | `PAY_PER_REQUEST` \| `PROVISIONED` | `PAY_PER_REQUEST` | `PAY_PER_REQUEST` is on-demand (no provisioned throughput). `PROVISIONED` emits `BillingMode: PROVISIONED` with `ProvisionedThroughput` on the table **and every GSI**, and unlocks Auto Scaling. Other value → exit `2`. |
+| `--rcu` | | integer ≥ 1 | `5` | Provisioned `ReadCapacityUnits` on the table and every GSI. **Only valid with `--billing-mode PROVISIONED`** — supplying it with `PAY_PER_REQUEST` is a loud reject (exit `2`). Non-integer / `< 1` → exit `2`. |
+| `--wcu` | | integer ≥ 1 | `5` | Provisioned `WriteCapacityUnits`. Same rules as `--rcu`. |
+| `--autoscale` | | boolean flag | off | Enable Application Auto Scaling (a `ScalableTarget` + `ScalingPolicy` pair for the table read/write and each GSI read/write). **Only valid with `--billing-mode PROVISIONED`** — with `PAY_PER_REQUEST` it is a loud reject (exit `2`). |
+| `--autoscale-min` | | integer ≥ 1 | `5` | Auto Scaling `MinCapacity` for every scalable target. Must be `<= --autoscale-max` (else exit `2`). Meaningful only with `--autoscale`. |
+| `--autoscale-max` | | integer ≥ 1 | `100` | Auto Scaling `MaxCapacity` for every scalable target. Must be `>= --autoscale-min`. |
+| `--autoscale-target` | | integer 1–100 | `70` | Target utilization % (`TargetValue` of the target-tracking policy). `> 100` → exit `2`. |
+| `--pitr` | | boolean flag | off | Point-in-time recovery (`PointInTimeRecoverySpecification` `PointInTimeRecoveryEnabled: true`). Omitted entirely when off (never emitted as `false`). |
+| `--table-class` | | `STANDARD` \| `STANDARD_INFREQUENT_ACCESS` | omitted | Emits `TableClass: <value>`. Omitted by default (AWS defaults to `STANDARD`). Other value → exit `2`. |
+| `--sse` | | boolean flag | off | Server-side encryption with the AWS-owned/managed key (`SSESpecification` `SSEEnabled: true` — no KMS key ARN / SSEType). Omitted entirely when off. |
+
+### Exit codes
+
+- `0` — generation succeeded.
+- `1` — unexpected error (e.g. the entry module threw). JSON error on stderr.
+- `2` — invalid arguments (missing / unreadable `--entry`, an invalid enum value,
+  a bad integer, or a loud reject below). JSON error on stderr
+  (`{"code":"INVALID_ARGS","message":…}`).
+
+## Generated resources
+
+For each **physical table** the model set maps onto, the template emits:
+
+- **`AWS::DynamoDB::Table`** with:
+  - `TableName` — a `!Ref` to a `TableName` parameter whose default is the
+    physical table name. (Multi-table designs get one parameter per table, keyed
+    by logical id.)
+  - `BillingMode` — `PAY_PER_REQUEST` (default) or `PROVISIONED`.
+  - `AttributeDefinitions` — **key attributes only**, every one `AttributeType: S`
+    (see derivation rules).
+  - `KeySchema` — `PK` (HASH) plus `SK` (RANGE) when the base key has a sort key.
+  - `GlobalSecondaryIndexes` — one per unioned GSI (omitted when there are none).
+  - `ProvisionedThroughput` — on the table **and every GSI**, only under
+    `PROVISIONED`.
+  - `StreamSpecification` — `NEW_AND_OLD_IMAGES`, only when streams are enabled.
+  - `TimeToLiveSpecification` — only when a `@ttl` field is declared (see TTL).
+  - `PointInTimeRecoverySpecification` / `TableClass` / `SSESpecification` — only
+    when the corresponding `--pitr` / `--table-class` / `--sse` option is given.
+- **`AWS::ApplicationAutoScaling::ScalableTarget` + `::ScalingPolicy`** — separate
+  top-level resources (not table properties), only under `--autoscale`.
+- **Parameters** — a `TableName` (String, default = physical name).
+- **Outputs** — the table name (`Ref`) and ARN (`Fn::GetAtt … Arn`), plus a
+  `StreamArn` output when streams are enabled.
+
+All resources, attribute definitions, and GSIs are emitted in a stable (sorted)
+order, so the output is byte-deterministic.
+
+## Derivation rules
+
+How a GraphDDB model set is mapped onto the physical DynamoDB shape:
+
+1. **One physical table per logical `table` name.** Entities that share a `table`
+   are single-table siblings and contribute to the same `AWS::DynamoDB::Table`.
+   The CFn logical resource id is `<AlphanumericLogicalName>Table` (e.g.
+   `UserPermissions` → `UserPermissionsTable`).
+2. **Physical key attribute names match the runtime exactly.** The base partition
+   key is always `PK`; the base sort key (when present) is `SK`. Each GSI uses
+   `<indexName>PK` and `<indexName>SK` (e.g. `GSI1PK` / `GSI1SK`).
+3. **`AttributeDefinitions` are key attributes only, all `AttributeType: S`.** It
+   contains `PK`, `SK` (when the table has a sort key), and each GSI's `<index>PK`
+   plus `<index>SK` (only for GSIs that have a sort key). **Non-key attributes are
+   never added** — in particular the TTL attribute is deliberately kept out
+   (CloudFormation rejects a non-key attribute there; `TimeToLiveSpecification`
+   references it legitimately).
+4. **The base `KeySchema` is `PK` (HASH) + `SK` (RANGE).** `SK` is included when
+   **any** entity on the table declares a sort key; a PK-only design omits `SK`
+   and its attribute definition.
+5. **GSIs are unioned by index name across all entities on the table.** The same
+   `IndexName` declared by several entities collapses to one GSI. Each GSI's
+   `KeySchema` is `<index>PK` (HASH) plus `<index>SK` (RANGE) — HASH-only when the
+   GSI has no sort key. `Projection.ProjectionType` is `ALL` (default) or
+   `KEYS_ONLY`. (Full GSI-union consistency — same sort-key presence across
+   entities, ≤ 20 GSIs — is enforced statically at model registration by the
+   `cfn-schema-consistency` linter rule; the emitter keeps a backstop that throws
+   on a surviving inconsistency.)
+6. **Streams (`--stream auto`) are derived from maintenance metadata.** `auto`
+   emits a stream when any registered entity uses stream-based maintenance
+   (`updateMode: 'stream'` — on a relation `write`, a versioned relation preset,
+   an `@aggregate` `write`, or a view's `@maintainedFrom`). This signal comes from
+   the metadata registry, not the manifest. The single view type is
+   `NEW_AND_OLD_IMAGES` (stream-based maintenance needs both images).
+7. **TTL is derived from a `@ttl` field.** When an entity on the table declares a
+   `@ttl` field, its physical attribute name is carried in the manifest and
+   rendered as `TimeToLiveSpecification { AttributeName, Enabled: true }`. DynamoDB
+   allows exactly one TTL attribute per table; the manifest build enforces this,
+   and the emitter backstops it by throwing if two entities on a table disagree.
+8. **Provisioned throughput + Auto Scaling are pure deploy-time settings** (from
+   CLI options, not derived from the model). Under `PROVISIONED`, the table and
+   every GSI carry `ProvisionedThroughput` (`--rcu` / `--wcu`, default 5/5).
+   Under `--autoscale`, each of the table read/write and every GSI read/write gets
+   a `ScalableTarget` (`ServiceNamespace: dynamodb`, the matching
+   `ScalableDimension`, an `Fn::Sub` `ResourceId` tying to the table's `Ref`,
+   `MinCapacity`/`MaxCapacity`, and the DynamoDB service-linked role ARN) plus a
+   `TargetTrackingScaling` `ScalingPolicy` (predefined
+   `DynamoDBRead/WriteCapacityUtilization` metric, `TargetValue`). So **N GSIs →
+   `2×(1+N)` scalable targets + `2×(1+N)` scaling policies.** GSI targets
+   additionally `DependsOn` the table.
+9. **PITR / TableClass / SSE are pure deploy-time settings**, each defaulting to
+   OFF/omitted — the emitter omits the whole property rather than emitting a
+   `false`/default value.
+
+## Scope boundary
+
+This command generates **only the GraphDDB-managed DynamoDB resources** — the
+table(s), their GSIs, and the DynamoDB-adjacent settings above (Streams, TTL,
+PITR, TableClass, SSE, provisioned throughput, and the Application Auto Scaling
+resources for that table). It does **not** generate IAM roles/policies for your
+application, Lambda functions, API Gateway, stream consumers, DynamoDB Streams
+event-source mappings, or any other application infrastructure. The one IAM
+reference it emits is the AWS **service-linked** role ARN that Application Auto
+Scaling uses (built with `Fn::Sub` so `${AWS::AccountId}` resolves at deploy
+time) — it does not create an IAM role. Compose the generated template into your
+own stack (nested stack / module) alongside those app resources.
+
+## Loud-reject / validation behaviors
+
+The CLI validates arguments before generation and **rejects loudly** (exit `2`,
+JSON `INVALID_ARGS` on stderr) rather than silently ignoring a nonsensical
+combination. The pure emitter keeps the same guards as an authoritative backstop
+(it throws). The loud rejects are:
+
+- **`--rcu` / `--wcu` with `PAY_PER_REQUEST`** — on-demand tables have no
+  provisioned capacity, so accepting the value would mislead you.
+- **`--autoscale` with `PAY_PER_REQUEST`** — Application Auto Scaling on an
+  on-demand table is invalid in AWS.
+- **`--autoscale-min` > `--autoscale-max`** — an invalid scalable-target range.
+- **`--autoscale-target` > 100** — it is a utilization percentage (1–100).
+- **A non-integer or `< 1` value for `--rcu` / `--wcu` / `--autoscale-*`.**
+- **An invalid enum value** for `--format` / `--gsi-projection` / `--stream` /
+  `--billing-mode` / `--table-class`.
+- **A missing `--entry`** (or one that cannot be imported).
+
+Inconsistent single-table declarations (a GSI with conflicting sort-key presence,
+or two `@ttl` attributes on one table) are caught statically by the
+`cfn-schema-consistency` linter at model registration; the emitter throws as a
+backstop if one survives to generation time.
+
+## Worked example
+
+The [`user-permissions`](../examples/user-permissions/) example is a single
+`UserPermissions` table whose entities (users, groups, memberships, permissions)
+share one physical table, with a `GSI1` **unioned** across `UserModel` (email
+lookup) and `GroupMembershipModel` (the inverted membership index). It exercises
+the base `PK`/`SK` key and a GSI union — it does **not** use streams, TTL, PITR,
+provisioned throughput, or auto scaling (those are opt-in flags / a `@ttl`
+decorator this model does not trigger).
+
+Generate it (the committed sample
+[`examples/user-permissions/cloudformation.yaml`](../examples/user-permissions/cloudformation.yaml)
+is produced by exactly this command, and a drift-guard test keeps it in sync):
+
+```bash
+graphddb generate cloudformation \
+  --entry examples/user-permissions/models.ts \
+  --out examples/user-permissions/cloudformation.yaml
+```
+
+The generated template (default YAML):
+
+```yaml
+AWSTemplateFormatVersion: "2010-09-09"
+Description: DynamoDB table(s) for a GraphDDB single-table design (generated by graphddb generate cloudformation).
+Parameters:
+  TableName:
+    Type: String
+    Default: UserPermissions
+    Description: Name of the UserPermissions DynamoDB table.
+Resources:
+  UserPermissionsTable:
+    Type: "AWS::DynamoDB::Table"
+    Properties:
+      TableName:
+        Ref: TableName
+      BillingMode: PAY_PER_REQUEST
+      AttributeDefinitions:
+        - AttributeName: PK
+          AttributeType: S
+        - AttributeName: SK
+          AttributeType: S
+        - AttributeName: GSI1PK
+          AttributeType: S
+        - AttributeName: GSI1SK
+          AttributeType: S
+      KeySchema:
+        - AttributeName: PK
+          KeyType: HASH
+        - AttributeName: SK
+          KeyType: RANGE
+      GlobalSecondaryIndexes:
+        - IndexName: GSI1
+          KeySchema:
+            - AttributeName: GSI1PK
+              KeyType: HASH
+            - AttributeName: GSI1SK
+              KeyType: RANGE
+          Projection:
+            ProjectionType: ALL
+Outputs:
+  TableName:
+    Description: Name of the UserPermissions DynamoDB table.
+    Value:
+      Ref: UserPermissionsTable
+  TableArn:
+    Description: ARN of the UserPermissions DynamoDB table.
+    Value:
+      "Fn::GetAtt":
+        - UserPermissionsTable
+        - Arn
+```
+
+### More option combinations
+
+```bash
+# JSON output.
+graphddb generate cloudformation --entry models.ts --format json
+
+# Key-only GSI projection.
+graphddb generate cloudformation --entry models.ts --gsi-projection KEYS_ONLY --out table.yaml
+
+# Force a DynamoDB stream (NEW_AND_OLD_IMAGES) on.
+graphddb generate cloudformation --entry models.ts --stream on --out table.yaml
+
+# Deploy-time hardening: PITR + SSE + Infrequent-Access table class.
+graphddb generate cloudformation --entry models.ts --pitr --sse --table-class STANDARD_INFREQUENT_ACCESS --out table.yaml
+
+# Provisioned billing with fixed throughput.
+graphddb generate cloudformation --entry models.ts --billing-mode PROVISIONED --rcu 10 --wcu 5 --out table.yaml
+
+# Provisioned billing + Application Auto Scaling.
+graphddb generate cloudformation --entry models.ts \
+  --billing-mode PROVISIONED --autoscale --autoscale-min 5 --autoscale-max 100 --autoscale-target 70 --out table.yaml
+```
+
+## Validating the output
+
+Real `aws cloudformation validate-template` requires AWS credentials and the
+`cloudformation:ValidateTemplate` IAM permission. Where those are unavailable
+(this project's CI is IAM-blocked, for example), an equivalent **structural**
+check is enough for a syntax gate: generate the template in both formats, parse
+each (YAML + JSON), assert the two agree, and assert the table structure. The
+repository wires this as `npm run validate:cfn` (see `scripts/validate-cfn.mjs`),
+which the CI runs as its "Validate generated CloudFormation (structural)" step.