cdk-glue-iceberg-table 0.13.0 → 0.14.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.jsii +97 -59
- package/README.md +57 -19
- package/dist/lib/iceberg/iceberg-partition-transform.js +1 -1
- package/dist/lib/iceberg/iceberg-table-properties.js +1 -1
- package/dist/lib/iceberg/iceberg-table-render.d.ts +4 -0
- package/dist/lib/iceberg/iceberg-table-render.js +41 -1
- package/dist/lib/iceberg/iceberg-table.d.ts +11 -0
- package/dist/lib/iceberg/iceberg-table.js +13 -2
- package/dist/lib/iceberg/iceberg-type.js +1 -1
- package/package.json +1 -1
package/.jsii
CHANGED
|
@@ -9751,7 +9751,7 @@
|
|
|
9751
9751
|
},
|
|
9752
9752
|
"name": "cdk-glue-iceberg-table",
|
|
9753
9753
|
"readme": {
|
|
9754
|
-
"markdown": "# cdk-glue-iceberg-table\n\n[](https://github.com/ksco92/arceus/actions/workflows/ci.yml)\n[](https://codecov.io/gh/ksco92/arceus)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://pypi.org/project/cdk-glue-iceberg-table/)\n[](https://constructs.dev/packages/cdk-glue-iceberg-table)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://github.com/ksco92/arceus/blob/main/LICENSE)\n\n**`cdk-glue-iceberg-table` is an AWS CDK L2 construct to create and\nevolve Apache Iceberg tables in the AWS Glue Data Catalog — schema and\npartition changes that survive CloudFormation updates, via cdk\ndeploy.** It supports Iceberg format version 3 (`variant`, nanosecond\ntimestamps, `geometry` / `geography`, `unknown`). It emits the exact\n`AWS::Glue::Table` / `OpenTableFormatInput` shape, so a single `cdk\ndeploy` creates, evolves, and destroys a table like any other CDK\nresource. No custom resource, no Lambda, no \"`cdk deploy` and then run\nthis SQL by hand\" two-step. Table changes (new columns, renames, drops,\nnew partition fields) land as a reviewed diff in a pull request and\napply through `cdk deploy`.\n\nStatus (October 2026): published on npm (TypeScript/JavaScript) and\nPyPI (Python), pre-1.0, with a public surface guarded by an end-to-end\nconsumer test on every PR plus real-AWS integration suites, including\nones for format version 3. The multi-language packages are generated\nfrom one TypeScript source with [jsii](https://aws.github.io/jsii/)\nand indexed on [Construct\nHub](https://constructs.dev/packages/cdk-glue-iceberg-table).\nDeveloped in the [`ksco92/arceus`](https://github.com/ksco92/arceus)\nmonorepo, which also holds a CDK demo app that dogfoods the construct\nagainst a real AWS account.\n\n### Why this exists\n\nAWS documents the CloudFormation shape for Iceberg tables ([AWS Big\nData blog, December 2025](https://aws.amazon.com/blogs/big-data/create-and-update-apache-iceberg-tables-with-partitions-in-the-aws-glue-data-catalog-using-the-aws-sdk-and-aws-cloudformation)),\nbut raw `AWS::Glue::Table` is a minefield: the CREATE succeeds and the\n*first* `Update` silently strips `table_type=ICEBERG`, after which\nevery Athena query fails with `HIVE_UNSUPPORTED_FORMAT`. The\nmotivating CDK issue is [aws/aws-cdk#29660](https://github.com/aws/aws-cdk/issues/29660);\n[`manmartgarc`'s comment](https://github.com/aws/aws-cdk/issues/29660#issuecomment-4411359248)\ndocuments the only working shape and the silent-corruption traps you\nhit by getting it slightly wrong. This construct implements that\nshape, refuses to emit the unsafe alternatives, and pins Iceberg field\nIDs so schema evolution never orphans existing data. It was the basis\nof an upstream proposal to land an `IcebergTable` in\n`@aws-cdk/aws-glue-alpha` ([aws/aws-cdk#37988](https://github.com/aws/aws-cdk/pull/37988)),\nclosed unmerged in September 2026.\n\n### How it compares\n\n| Approach | Declarative IaC (in the PR) | In-place schema + partition evolution | Prevents the silent-corruption footguns | Typed API + synth-time validation |\n| --- | --- | --- | --- | --- |\n| Raw `CfnTable` (L1) | ✅ | ⚠️ only if you hand-write the exact `OpenTableFormatInput` shape | ❌ you own both footguns | ❌ |\n| CDK custom resource (Lambda + Glue SDK) | ⚠️ via a custom resource you maintain | ⚠️ you write the diff logic | ⚠️ your code's responsibility | ❌ |\n| Spark SQL DDL, or Athena SQL DDL, at job runtime | ❌ imperative, outside the PR | ✅ but outside CloudFormation | n/a | ❌ |\n| **`cdk-glue-iceberg-table`** | ✅ | ✅ via `cdk deploy` | ✅ both prevented by construction | ✅ |\n\n## Install\n\nTypeScript / JavaScript (npm):\n\n```bash\nnpm install cdk-glue-iceberg-table\n```\n\nPython (PyPI):\n\n```bash\npip install cdk-glue-iceberg-table\n```\n\nPeer dependencies (your CDK app must already have these). This\nrelease requires `aws-cdk-lib` 2.273.0 or newer and `constructs`\n10.8.1 or newer, plus Node.js 22.14 or newer for both languages (the\nPython package runs under Node.js through jsii).\n\nFor TypeScript / JavaScript:\n\n```bash\nnpm install aws-cdk-lib@^2.273.0 constructs@^10.8.1\n```\n\nFor Python:\n\n```bash\npip install 'aws-cdk-lib>=2.273.0' 'constructs>=10.8.1'\n```\n\n## Use\n\n### TypeScript\n\n```ts\nimport { Bucket } from 'aws-cdk-lib/aws-s3';\nimport { Database } from 'aws-cdk-lib/aws-glue';\nimport {\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n IcebergPartitionTransform,\n} from 'cdk-glue-iceberg-table';\n\nconst bucket = new Bucket(this, 'Warehouse');\nconst db = new Database(this, 'Db', { databaseName: 'analytics' });\n\nnew IcebergTable(this, 'OrdersTable', {\n database: db,\n tableName: 'orders',\n location: `s3://${bucket.bucketName}/analytics/orders/`,\n columns: [\n { name: 'order_id', type: IcebergType.LONG, required: true, id: 1 },\n { name: 'customer_id', type: IcebergType.LONG, required: true, id: 2 },\n { name: 'placed_at', type: IcebergType.TIMESTAMPTZ, required: true, id: 3 },\n ],\n partitionSpec: [\n { sourceColumn: 'placed_at', transform: IcebergPartitionTransform.DAY, fieldId: 1000 },\n { sourceColumn: 'customer_id', transform: IcebergPartitionTransform.bucket(16), fieldId: 1001 },\n ],\n identifierFieldNames: ['order_id'],\n formatVersion: IcebergFormatVersion.V2,\n});\n```\n\n### Python\n\nThe same table in Python — the API mirrors TypeScript with\n`snake_case` props and `PascalCase` types:\n\n```python\nfrom aws_cdk.aws_s3 import Bucket\nfrom aws_cdk.aws_glue import Database\nfrom cdk_glue_iceberg_table import (\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n IcebergPartitionTransform,\n)\n\nbucket = Bucket(self, \"Warehouse\")\ndb = Database(self, \"Db\", database_name=\"analytics\")\n\nIcebergTable(self, \"OrdersTable\",\n database=db,\n table_name=\"orders\",\n location=f\"s3://{bucket.bucket_name}/analytics/orders/\",\n columns=[\n {\"name\": \"order_id\", \"type\": IcebergType.LONG, \"required\": True, \"id\": 1},\n {\"name\": \"customer_id\", \"type\": IcebergType.LONG, \"required\": True, \"id\": 2},\n {\"name\": \"placed_at\", \"type\": IcebergType.TIMESTAMPTZ, \"required\": True, \"id\": 3},\n ],\n partition_spec=[\n {\"source_column\": \"placed_at\", \"transform\": IcebergPartitionTransform.DAY, \"field_id\": 1000},\n {\"source_column\": \"customer_id\", \"transform\": IcebergPartitionTransform.bucket(16), \"field_id\": 1001},\n ],\n identifier_field_names=[\"order_id\"],\n format_version=IcebergFormatVersion.V2,\n)\n```\n\nConsumer-facing reference sections:\n\n- [How it compares](#how-it-compares) — `cdk-glue-iceberg-table` vs raw `CfnTable`, custom resources, and runtime SQL DDL.\n- [Using `IcebergTable`](#using-icebergtable) — full API reference with examples.\n- [Format version 3](#format-version-3) — the six v3 types, which engines read them, and the accepted limits.\n- [Two footguns the construct prevents](#two-footguns-the-construct-prevents) — the silent-corruption traps that motivated this construct.\n- [Known limitations](#known-limitations) — what the construct does and doesn't enforce.\n- [FAQ](#faq) — common \"how do I … in CDK / CloudFormation\" questions.\n\n## Exported surface\n\nThe package entry point re-exports the main types you import from\n`cdk-glue-iceberg-table`. In TypeScript / JavaScript the compiled\nsurface lives under `dist/lib/iceberg/` after `npm install`; in Python\nthe same surface is imported from the `cdk_glue_iceberg_table` module\nafter `pip install`:\n\n- **`IcebergTable`** — the L2 construct itself, with `grantRead` / `grantWrite` / `grantReadWrite` and the `fromIcebergTableAttributes(...)` import factory.\n- **`IcebergType`** — primitive types, the v3 constants `UNKNOWN`, `TIMESTAMP_NS`, `TIMESTAMPTZ_NS`, and `VARIANT`, and the `list` / `map` / `struct` / `decimal` / `fixed` / `geometry(crs?)` / `geography(crs?, algorithm?)` factories. `list` takes an `IcebergListDefinition`, `map` an `IcebergMapDefinition`, and `struct` a list of `IcebergStructFieldDefinition`, each carrying the nested field ids. Renders to the string Glue's `IcebergStructField.type` expects.\n- **`IcebergPartitionTransform`** — identity / `bucket(N)` / `truncate(W)` / year / month / day / hour / void. Each transform validates against the source column type at synth time.\n- **`IcebergDataFormat`** (parquet/orc/avro, default parquet), **`IcebergFormatVersion`** (v1/v2/v3, required — set explicitly per table; Athena cannot query v3 tables), **`IcebergSortDirection`**, **`IcebergNullOrder`**, **`IcebergEdgeAlgorithm`** (`SPHERICAL`, `VINCENTY`, `THOMAS`, `ANDOYER`, `KARNEY`; the second argument of `IcebergType.geography`), plus a `tableProperties` validator that catches misconfigured properties before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).\n\n## Using `IcebergTable`\n\nA minimal table:\n\n```typescript\nimport {\n Bucket,\n} from 'aws-cdk-lib/aws-s3';\nimport {\n Database,\n} from 'aws-cdk-lib/aws-glue';\nimport {\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nconst bucket = new Bucket(this, 'Warehouse');\n\nconst db = new Database(this, 'Db', {\n databaseName: 'analytics',\n});\n\nnew IcebergTable(this, 'Users', {\n database: db,\n tableName: 'users',\n columns: [\n {\n name: 'user_id',\n type: IcebergType.LONG,\n required: true,\n id: 1,\n },\n {\n name: 'email',\n type: IcebergType.STRING,\n required: true,\n id: 2,\n },\n {\n name: 'signed_up_at',\n type: IcebergType.TIMESTAMPTZ,\n required: true,\n id: 3,\n },\n ],\n location: `s3://${bucket.bucketName}/analytics/users/`,\n formatVersion: IcebergFormatVersion.V2,\n});\n```\n\nEvery top-level column must declare an `id`: a stable, unique,\npositive integer you pick once and never reuse or reassign. Because\nIceberg data files reference fields by id rather than by name, a fixed\nid is what lets you add, rename, and drop columns across deploys\nwithout corrupting old data — see\n[How do I evolve an Iceberg table schema](#how-do-i-evolve-an-iceberg-table-schema-add-rename-or-drop-a-column-in-cloudformation).\nOnce a column is dropped, leave its id retired; never hand it to a new\ncolumn. Nested fields follow the same rule: every struct field takes\nan `id`, every list an `elementId`, and every map a `keyId` and a\n`valueId`. All ids in a schema, top-level and nested, must be distinct\nintegers from 1 to 2147483447.\n\nA table that exercises most of the surface (partitions, sort order,\nnested types, identifier fields, table properties, removal policy):\n\n```typescript\nimport {\n RemovalPolicy,\n} from 'aws-cdk-lib';\nimport {\n Database,\n} from 'aws-cdk-lib/aws-glue';\nimport {\n IcebergDataFormat,\n IcebergFormatVersion,\n IcebergNullOrder,\n IcebergPartitionTransform,\n IcebergSortDirection,\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nnew IcebergTable(this, 'OrdersTable', {\n database: db,\n tableName: 'orders',\n comment: 'Demo Iceberg orders table — exercises partitions, sort order, and merge-on-read.',\n columns: [\n {\n name: 'order_id',\n type: IcebergType.LONG,\n required: true,\n id: 1,\n },\n {\n name: 'customer_id',\n type: IcebergType.LONG,\n required: true,\n id: 2,\n },\n {\n name: 'order_amount',\n type: IcebergType.decimal(12, 2),\n required: true,\n id: 3,\n },\n {\n name: 'currency',\n type: IcebergType.STRING,\n required: true,\n id: 4,\n },\n {\n name: 'placed_at',\n type: IcebergType.TIMESTAMPTZ,\n required: true,\n id: 5,\n },\n {\n name: 'tags',\n type: IcebergType.list({\n element: IcebergType.STRING,\n elementId: 9,\n }),\n id: 6,\n },\n {\n name: 'shipping_address',\n type: IcebergType.struct([\n {\n id: 10,\n name: 'line1',\n type: IcebergType.STRING,\n required: true,\n },\n {\n id: 11,\n name: 'city',\n type: IcebergType.STRING,\n required: true,\n },\n {\n id: 12,\n name: 'country',\n type: IcebergType.STRING,\n required: true,\n },\n {\n id: 13,\n name: 'postal_code',\n type: IcebergType.STRING,\n },\n ]),\n id: 7,\n },\n {\n name: 'metadata',\n type: IcebergType.map({\n key: IcebergType.STRING,\n keyId: 14,\n value: IcebergType.STRING,\n valueId: 15,\n }),\n id: 8,\n },\n ],\n location: `s3://${bucket.bucketName}/analytics/orders/`,\n partitionSpec: [\n {\n sourceColumn: 'placed_at',\n transform: IcebergPartitionTransform.DAY,\n fieldId: 1000,\n },\n {\n sourceColumn: 'customer_id',\n transform: IcebergPartitionTransform.bucket(16),\n fieldId: 1001,\n },\n ],\n sortOrder: [\n {\n sourceColumn: 'placed_at',\n direction: IcebergSortDirection.ASC,\n nullOrder: IcebergNullOrder.NULLS_LAST,\n },\n {\n sourceColumn: 'order_id',\n direction: IcebergSortDirection.ASC,\n },\n ],\n identifierFieldNames: [\n 'order_id',\n ],\n dataFormat: IcebergDataFormat.PARQUET,\n formatVersion: IcebergFormatVersion.V2,\n tableProperties: {\n 'write.parquet.compression-codec': 'zstd',\n 'write.delete.mode': 'merge-on-read',\n 'write.update.mode': 'merge-on-read',\n 'write.merge.mode': 'merge-on-read',\n 'write.target-file-size-bytes': '134217728',\n 'history.expire.min-snapshots-to-keep': '5',\n 'gc.enabled': 'true',\n },\n removalPolicy: RemovalPolicy.DESTROY,\n});\n```\n\nEach partition field also pins a `fieldId`: an integer of 1000 or\nabove (the Iceberg spec reserves lower ids for schema columns), unique\nacross the table's entire partition-spec history. The discipline is\nthe inverse of the column-id rule. A column keeps its id forever; a\npartition field keeps its id only while it stays untouched. Change\nanything about a field — swap `month` for `day`, point it at a\ndifferent source column — and it becomes a *new* field that needs a\nfresh, never-before-used id, while the old id retires with the old\nspec. Reusing a retired id (or reassigning an id to a different field)\nproduces a table that Athena still reads but Spark and Trino reject\nwith `ValidationException: Conflicting partition fields`.\n\n`identifierFieldNames` lists the columns that together identify a row\nand renders as the schema's `identifier-field-ids`. Each one must be a\n`required: true` column (Glue rejects an optional one), and `cdk synth`\nrejects an identifier column of type float, double, list, map, struct,\nunknown, variant, geometry or geography. The construct prefixes each\nidentifier column's `doc` with `(PK) `, so a `doc` you write must not\nstart with `(PK) `; see [Evolving schema and\npartitions](#evolving-schema-and-partitions).\n\nThe resulting Iceberg `metadata.json` for this table contains every\nfeature you set:\n\n```json\n{\n \"format-version\": 2,\n \"table-uuid\": \"39a948f9-...\",\n \"current-schema-id\": 0,\n \"schemas\": [\n {\n \"schema-id\": 0,\n \"identifier-field-ids\": [1],\n \"fields\": [\n { \"id\": 1, \"name\": \"order_id\", \"required\": true, \"type\": \"long\", \"doc\": \"(PK) \" },\n { \"id\": 2, \"name\": \"customer_id\", \"required\": true, \"type\": \"long\" },\n { \"id\": 3, \"name\": \"order_amount\", \"required\": true, \"type\": \"decimal(12, 2)\" },\n { \"id\": 4, \"name\": \"currency\", \"required\": true, \"type\": \"string\" },\n { \"id\": 5, \"name\": \"placed_at\", \"required\": true, \"type\": \"timestamptz\" },\n { \"id\": 6, \"name\": \"tags\", \"required\": false,\n \"type\": { \"type\": \"list\", \"element-id\": 9, \"element\": \"string\", \"element-required\": false } },\n { \"id\": 7, \"name\": \"shipping_address\", \"required\": false,\n \"type\": { \"type\": \"struct\", \"fields\": [\n { \"id\": 10, \"name\": \"line1\", \"required\": true, \"type\": \"string\" },\n { \"id\": 11, \"name\": \"city\", \"required\": true, \"type\": \"string\" },\n { \"id\": 12, \"name\": \"country\", \"required\": true, \"type\": \"string\" },\n { \"id\": 13, \"name\": \"postal_code\", \"required\": false, \"type\": \"string\" }\n ] } },\n { \"id\": 8, \"name\": \"metadata\", \"required\": false,\n \"type\": { \"type\": \"map\", \"key-id\": 14, \"key\": \"string\", \"value-id\": 15,\n \"value-required\": false, \"value\": \"string\" } }\n ]\n }\n ],\n \"partition-specs\": [\n { \"spec-id\": 0, \"fields\": [\n { \"name\": \"placed_at_day\", \"transform\": \"day\", \"source-id\": 5, \"field-id\": 1000 },\n { \"name\": \"customer_id_bucket\", \"transform\": \"bucket[16]\", \"source-id\": 2, \"field-id\": 1001 }\n ]}\n ],\n \"sort-orders\": [\n { \"order-id\": 1, \"fields\": [\n { \"transform\": \"identity\", \"source-id\": 5, \"direction\": \"asc\", \"null-order\": \"nulls-last\" },\n { \"transform\": \"identity\", \"source-id\": 1, \"direction\": \"asc\", \"null-order\": \"nulls-last\" }\n ]}\n ],\n \"properties\": {\n \"format-version\": \"2\",\n \"write.format.default\": \"parquet\",\n \"write.parquet.compression-codec\": \"zstd\",\n \"write.merge.mode\": \"merge-on-read\",\n \"write.update.mode\": \"merge-on-read\",\n \"write.delete.mode\": \"merge-on-read\",\n \"write.target-file-size-bytes\": \"134217728\",\n \"history.expire.min-snapshots-to-keep\": \"5\",\n \"gc.enabled\": \"true\",\n \"comment\": \"Demo Iceberg orders table — exercises partitions, sort order, and merge-on-read.\"\n }\n}\n```\n\n### Format version 3\n\nSet `formatVersion: IcebergFormatVersion.V3` to create a v3 table. The\nsix v3 types (below) are only accepted on a V3 table, at any nesting\ndepth: inside a struct, as a list element, or as a map key or value.\nSynth fails on a V1 or V2 table that uses one.\n\n```typescript\nimport {\n IcebergEdgeAlgorithm,\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nnew IcebergTable(this, 'Events', {\n database: db,\n tableName: 'events',\n location: `s3://${bucket.bucketName}/analytics/events/`,\n columns: [\n { name: 'occurred_at', type: IcebergType.TIMESTAMP_NS, id: 1 },\n { name: 'payload', type: IcebergType.VARIANT, id: 2 },\n {\n name: 'origin',\n type: IcebergType.geography('OGC:CRS84', IcebergEdgeAlgorithm.SPHERICAL),\n id: 3,\n },\n ],\n formatVersion: IcebergFormatVersion.V3,\n});\n```\n\nThe same table in Python:\n\n```python\nfrom cdk_glue_iceberg_table import (\n IcebergEdgeAlgorithm,\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n)\n\nIcebergTable(self, \"Events\",\n database=db,\n table_name=\"events\",\n location=f\"s3://{bucket.bucket_name}/analytics/events/\",\n columns=[\n {\"name\": \"occurred_at\", \"type\": IcebergType.TIMESTAMP_NS, \"id\": 1},\n {\"name\": \"payload\", \"type\": IcebergType.VARIANT, \"id\": 2},\n {\n \"name\": \"origin\",\n \"type\": IcebergType.geography(\"OGC:CRS84\", IcebergEdgeAlgorithm.SPHERICAL),\n \"id\": 3,\n },\n ],\n format_version=IcebergFormatVersion.V3,\n)\n```\n\nThis table has a `timestamp_ns` column; see [Known limitations](#known-limitations)\nbefore reading it from upstream Spark.\n\nThe v3 types, with the rules `cdk synth` enforces:\n\n| Type | Factory | Rendered string | Partition and identifier rules |\n| --- | --- | --- | --- |\n| unknown | `IcebergType.UNKNOWN` | `unknown` | Must be optional, as a column, list element, or map value; never a map key. Partition source only with `void`. Not an identifier field. |\n| timestamp with nanoseconds | `IcebergType.TIMESTAMP_NS` | `timestamp_ns` | `identity`, `year`, `month`, `day`, `hour`, `bucket(N)`, `void`; not `truncate(W)`. Allowed as an identifier field. |\n| timestamptz with nanoseconds | `IcebergType.TIMESTAMPTZ_NS` | `timestamptz_ns` | Same as `timestamp_ns`. |\n| variant | `IcebergType.VARIANT` | `variant` | Never a partition or sort source, `void` included. Not an identifier field. |\n| geometry | `IcebergType.geometry(crs?)` | `geometry`, or `geometry(<crs>)` when a CRS is given | Partition source only with `void`. Not an identifier field. |\n| geography | `IcebergType.geography(crs?, algorithm?)` | `geography`, `geography(<crs>)`, or `geography(<crs>, <algorithm>)` when an algorithm is given, such as `geography(OGC:CRS84, spherical)` | Same as `geometry`. |\n\nA geometry or geography `crs` is an identifier such as `OGC:CRS84` or\n`EPSG:3857`; whitespace, parentheses, braces, and commas are rejected.\nWhen `algorithm` is set without a `crs`, the rendered string spells out\n`OGC:CRS84`.\n\nWhich engines read a v3 table created by the construct:\n\n| Engine | v3 tables | Notes |\n| --- | --- | --- |\n| Spark 4.1 + Iceberg 1.12 (the integration-test runner) | Yes | Reads and writes tables the construct creates, including deletion vectors, row lineage, `variant`, `unknown`, and `geometry` / `geography`, except tables with nanosecond-timestamp columns or a geometry CRS other than `OGC:CRS84` (checked by `scripts/spark/verify_v3_dml.py`; details in [Known limitations](#known-limitations)). |\n| AWS Glue 6.0 | Documented | The [AWS Glue versions](https://docs.aws.amazon.com/glue/latest/dg/release-notes.html) page lists \"Iceberg format version 3 with VARIANT data type, deletion vectors, row lineage tracking, nanosecond timestamps, and geo types\". The integration suites do not run on Glue 6.0. |\n| Amazon EMR Spark 8.1.0 (Spark 4.1.1, Iceberg 1.11.0-amzn-0) | Documented | The [emr-spark-8.1.0 release notes](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark810-release.html) state \"Amazon EMR Spark 8.1.0 extends Iceberg v3 spec support for Spark workloads\". The integration suites do not run on EMR 8.1. |\n| Athena SQL | No | The [Athena Iceberg page](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg.html) states \"Athena only creates and operates on Iceberg v2 tables.\" |\n\n### Granting access\n\n```typescript\ntable.grantRead(role); // Glue read + S3 read on the table's prefix\ntable.grantWrite(role); // Glue write + S3 write\ntable.grantReadWrite(role);\n```\n\nThe `grant*` helpers issue **IAM grants only**. Under Lake Formation\nyou still add the matching `SELECT` / `INSERT` / `DELETE` LF grants on\ntop of the construct's IAM grants for Athena queries to succeed.\n\n### Importing an existing table\n\n```typescript\nconst existing = IcebergTable.fromIcebergTableAttributes(this, 'Orders', {\n database: db,\n tableName: 'orders',\n location: 's3://my-bucket/analytics/orders/',\n});\nexisting.grantRead(role);\n```\n\n### Evolving schema and partitions\n\nChange the `columns` array (or `partitionSpec`) and run `cdk deploy`\nagain. The construct passes the new schema to Glue's `UpdateTable`,\nwhich writes a new `metadata.json` with a new `schema-id`; existing\ndata files stay readable because each column's `id` is pinned and\nnever reused. Adds, renames (same `id`, new `name`), and drops all\nflow through `cdk deploy` alone — no out-of-band SQL DDL.\n\nNested fields evolve the same way. Each struct field, list element,\nmap key, and map value keeps the id you gave it (the rule is under\n[Using `IcebergTable`](#using-icebergtable)), so adding a column or a\nstruct field anywhere in the schema leaves every other id unchanged. A\nnew nested field takes an id no field of the table has ever used:\n\n```typescript\ntype: IcebergType.struct([\n { id: 10, name: 'line1', type: IcebergType.STRING },\n { id: 16, name: 'line2', type: IcebergType.STRING }, // new field, never-used id\n { id: 11, name: 'city', type: IcebergType.STRING },\n]),\n```\n\nUpgrading from a version before 0.12.0, which numbered nested fields\nitself: pin each nested field to the id the deployed table already\nhas, or the first deploy renumbers it and existing data files read\nunder the wrong field. Read the ids from the `element-id`, `key-id`,\n`value-id`, and struct field `id` values in the previous version's\nsynthesized template (`cdk.out/<Stack>.template.json`) or in the\ntable's current `metadata.json`. A nested type object shared by two\ncolumns now fails synth with a duplicate id; give each column its own\nnested type with its own ids. The list element and map value\nnullability default also changed: `elementRequired` and\n`valueRequired` now default to `false`. Earlier versions made list\nelements required by default and map values required unless the third\nargument of `map` was `false`; set `elementRequired: true` or\n`valueRequired: true` wherever the deployed schema has\n`\"element-required\": true` or `\"value-required\": true`.\n\nGlue's `UpdateTable` reuses an earlier schema whose fields (name,\ntype, `required`, `doc`) match the request, together with that\nschema's `identifier-field-ids`. The construct therefore prefixes the\n`doc` of every identifier column with `(PK) ` (just `(PK) ` when the\ncolumn has no `doc`), shown as the column comment, so every\nidentifier change is a field change. On a table whose schema history\nwas written with the marker, identifier changes flow through `cdk\ndeploy` like column changes.\n\nPartition evolution flows the same way, with one extra rule on\n`fieldId`: a changed partition field is a new partition field. To move\na table from `month(placed_at)` to `day(placed_at)`, drop the month\nentry and add a day entry under a fresh `fieldId` — never recycle the\nmonth field's id:\n\n```typescript\npartitionSpec: [\n // was: { sourceColumn: 'placed_at', transform: IcebergPartitionTransform.MONTH, fieldId: 1000 },\n {\n sourceColumn: 'placed_at',\n transform: IcebergPartitionTransform.DAY,\n fieldId: 1001,\n },\n],\n```\n\nOld data files written under the month spec stay readable; new writes\npartition by day. Reusing `1000` for the day field instead corrupts\nthe spec history for strict engines (Spark, Trino) while Athena keeps\nworking — the failure only surfaces at Spark runtime.\n\n### Inserting and querying\n\n```sql\n-- INSERT into the orders table\nINSERT INTO sample_database.orders VALUES\n (1001, 5001, DECIMAL '149.99', 'USD',\n TIMESTAMP '2026-05-20 09:15:00 UTC',\n ARRAY['holiday-promo','first-order'],\n CAST(ROW('1 Infinite Loop','Cupertino','US','95014')\n AS ROW(line1 VARCHAR,city VARCHAR,country VARCHAR,postal_code VARCHAR)),\n MAP(ARRAY['channel','utm'], ARRAY['web','google'])),\n -- ... more rows\n;\n\n-- merge-on-read DELETE (only legal because we chose v2 + merge-on-read mode)\nDELETE FROM sample_database.orders WHERE order_id = 1003;\n\n-- merge-on-read UPDATE\nUPDATE sample_database.orders SET currency = 'GBP' WHERE customer_id = 5002;\n\n-- SELECT\nSELECT customer_id, SUM(order_amount) AS total\n FROM sample_database.orders\n GROUP BY 1\n ORDER BY 2 DESC;\n```\n\n## Two footguns the construct prevents\n\n### Footgun #1 — schema under `storageDescriptor.columns`\n\nThe CREATE succeeds but the first UPDATE silently strips\n`table_type=ICEBERG` from the table's Glue parameters, and Athena\nqueries after that fail with `HIVE_UNSUPPORTED_FORMAT`.\n\n```typescript\n// DON'T DO THIS — what most StackOverflow / re:Post examples show\nnew CfnTable(this, 'OrdersBad', {\n catalogId: this.account,\n databaseName: 'analytics',\n tableInput: {\n name: 'orders',\n tableType: 'EXTERNAL_TABLE',\n parameters: {\n table_type: 'ICEBERG',\n },\n storageDescriptor: {\n location: 's3://.../orders/',\n columns: [\n /* ... */\n ],\n },\n },\n openTableFormatInput: {\n icebergInput: {\n metadataOperation: 'CREATE',\n version: '2',\n },\n },\n});\n```\n\n`IcebergTable` instead always emits schema/partitions/sort/properties\nunder `openTableFormatInput.icebergInput.icebergTableInput`, never\nunder `storageDescriptor`.\n\n### Footgun #2 — `tableInput` co-present with `openTableFormatInput`\n\nEven setting just `tableInput: { name: 'foo' }` next to\n`openTableFormatInput` returns\n`\"Table metadata is expected only via TableInput or via IcebergTableInputProperties inside OpenTableFormatInput\"`.\nThe construct never emits `tableInput`; the table-level comment goes\ninto `tableProperties['comment']`, which lives inside\n`icebergTableInput.properties`.\n\n(There is a third footgun, field-id reuse after a column drop, that\nthe construct does **not** prevent. See the next section.)\n\n## Known limitations\n\n- **Field-id reuse is not detected across deploys.** This applies to nested ids (struct field, list element, map key and value) as well as column ids. If you drop a column with `id = 5` and then add a different column with `id = 5` in a later deploy, Glue accepts the UPDATE and Iceberg's metadata silently violates the \"never reuse a retired id\" invariant. Readers projecting old snapshots will surface deleted data under the new field's name. The construct enforces uniqueness **within one deploy** (`duplicate field id N` validator), but it doesn't compare against the live table state. The safe workflow is to treat dropped ids as retired forever.\n- **Partition field-id reuse is not detected across deploys either.** The same live-state blindness applies to partition `fieldId`s: the construct validates them within one deploy (integer >= 1000, unique in the spec), but it cannot tell that the id you pinned already appeared in an earlier spec of the deployed table. The invariant partition ids must uphold is stricter than the column one — unique across the table's **entire spec history**, so a changed field (different transform or source column) must take a fresh id rather than keep its old one. Violate it and strict engines (Spark, Trino) reject the table with `Conflicting partition fields` while Athena keeps answering, masking the break until Spark runtime. The safe workflow: treat every id that has ever appeared in any spec as retired, and allocate new fields the next never-used id. A partition change needs a new source column or transform; Glue ignores a `fieldId`-only change.\n- **`last-partition-id` is written by Glue, not by the construct.** The construct supplies the spec's field ids; Glue's `UpdateTable` computes the table's `last-partition-id` counter when it writes the new metadata. The repo's integration test asserts the counter keeps up with the highest supplied field id after a partition evolution, but engines that later evolve the spec themselves (e.g. Spark `ALTER TABLE … ADD PARTITION FIELD`) allocate from that counter — if you mix CDK-managed and engine-managed partition evolution, verify the counter in `metadata.json` before trusting engine-side changes.\n- **Removing the last identifier from a table created with identifiers before 0.11.0 can bring the old ids back.** That table's history holds an unmarked schema with ids, and Glue reuses it when the rendered fields match it again. Change a column `doc` in the same deploy that removes the last identifier.\n- **CREATE-only metadata operation.** The CFN `IcebergInput.metadataOperation` only accepts `CREATE`; the construct always emits that. Subsequent deploys use Glue's normal `UpdateTable` path, which writes new Iceberg metadata in-place.\n- **Format version is immutable after CREATE.** Glue ignores a V2 to V3 `formatVersion` change, though `properties.format-version` then reads `3`. Keep `formatVersion` at the table's real version, or destroy + recreate the table to change it. An engine can upgrade a v2 table to v3 in place with `ALTER TABLE ... SET TBLPROPERTIES ('format-version' = '3')`; after that, set `formatVersion` to `V3` so the next deploy declares the version the table already has.\n- **`merge-on-read` requires v2 or later.** The construct rejects `write.{delete,update,merge}.mode = merge-on-read` on a v1 table at synth time.\n- **Column defaults are not available.** Iceberg v3 `initial-default` and `write-default` cannot be declared through CloudFormation: the `AWS::Glue::Table` resource schema has no `InitialDefault` or `WriteDefault` property. Revisit when that schema gains the two properties.\n- **Nanosecond timestamps are catalog-only for upstream Spark 4.1.** A table with any `timestamp_ns` or `timestamptz_ns` column fails to open in upstream Spark 4.1 with Iceberg 1.12. The integration suite for these columns asserts from `metadata.json` only. The [engine matrix](#format-version-3) lists the engines that document support.\n- **Geometry CRS other than `OGC:CRS84` is catalog-only for upstream Spark 4.1.** Spark 4.1 accepts `OGC:CRS84`, or bare `geometry`, as the geometry CRS; a table declaring another CRS, such as `EPSG:3857`, cannot be read there.\n- **Athena DDL features that don't surface through CFN** (e.g. `ALTER TABLE WRITE ORDERED BY`, `ALTER TABLE … SET LOCATION`, `bucketed_by` / `bucket_count` Hive clauses) are not exposed. Use `IcebergPartitionTransform.bucket(N)` instead of Hive bucketing.\n- **Dropping a partition column requires a `void` intermediate per the Iceberg spec**, and the CFN `OpenTableFormatInput` cannot express that. The construct accepts the change, but Athena queries against the result will fail with `Type cannot be null`. The safe pattern is to drop partitions that source from a column while keeping that column in the schema, and only drop a column once it is no longer partitioning anything.\n\n## FAQ\n\n### How do I create an Apache Iceberg table in AWS CDK?\n\nInstall `cdk-glue-iceberg-table`, then declare an `IcebergTable` with a\ndatabase, a column list, and an S3 `location` — see [Use](#use).\n`cdk deploy` creates the Glue Data Catalog table and writes the\nIceberg `metadata.json` under your S3 prefix. No custom resource or\nLambda is involved; it is a plain `AWS::Glue::Table` in your\nCloudFormation template.\n\n### How do I evolve an Iceberg table schema (add, rename, or drop a column) in CloudFormation?\n\nChange the `columns` array and run `cdk deploy` again. The construct\npasses the new schema to Glue's `UpdateTable`, which writes a new\n`metadata.json` with a new `schema-id`; existing data files stay\nreadable because each column's `id` is pinned and never reused.\nPartition changes work the same way, except a changed partition field\ntakes a fresh `fieldId` instead of keeping its old one. See [Evolving\nschema and partitions](#evolving-schema-and-partitions).\n\n### Does CloudFormation support Iceberg tables natively?\n\nYes, through `AWS::Glue::Table` with `OpenTableFormatInput.IcebergInput`,\ndocumented by AWS in [December 2025](https://aws.amazon.com/blogs/big-data/create-and-update-apache-iceberg-tables-with-partitions-in-the-aws-glue-data-catalog-using-the-aws-sdk-and-aws-cloudformation).\nThe catch is that the raw shape corrupts the table on the first\n`Update` if you place schema under `storageDescriptor.columns` or set\n`tableInput` alongside `openTableFormatInput`. This construct only ever\nemits the safe shape and gives you no way to express the unsafe ones.\nSee [Two footguns the construct prevents](#two-footguns-the-construct-prevents).\n\n### What is the difference between cdk-glue-iceberg-table and a raw CfnTable?\n\nA raw `CfnTable` makes you hand-write the `OpenTableFormatInput` JSON\nand own both silent-corruption footguns. `cdk-glue-iceberg-table` gives\nyou a typed `IcebergType` / `IcebergPartitionTransform` API,\nsynth-time validation (partition transforms checked against column\ntypes, format/version mismatches caught before deploy), pinned field\nIDs for safe evolution, and `grantRead` / `grantWrite` helpers. See\n[How it compares](#how-it-compares).\n\n### How do I create a partitioned Iceberg table in CDK?\n\nPass a `partitionSpec` of `IcebergPartitionTransform` entries —\n`identity`, `bucket(N)`, `truncate(W)`, `year`, `month`, `day`, `hour`,\nor `void` — each pinned to a `fieldId` of 1000 or above. Transforms are\nvalidated against their source column's type at synth time. See\n[Using `IcebergTable`](#using-icebergtable).\n\n### Does it work with Athena and Lake Formation?\n\nYes for v2 tables, no for v3 (see the engine matrix in [Format\nversion 3](#format-version-3)). On v2, Athena reads and writes,\nincluding merge-on-read `INSERT` / `UPDATE` / `DELETE` / `MERGE`, time\ntravel, `OPTIMIZE`, and `VACUUM`. The demo app in the\n[`arceus`](https://github.com/ksco92/arceus) repo registers its tables\nwith Lake Formation and exercises the same operations through Spark in\nits integration suite. The construct's `grant*` helpers issue IAM\ngrants; under Lake Formation you still add the matching `SELECT` /\n`INSERT` LF grants. See [Granting access](#granting-access).\n\n### Can I use Iceberg v2 merge-on-read (row-level UPDATE and DELETE)?\n\nYes — set `formatVersion: IcebergFormatVersion.V2` and the\n`write.{delete,update,merge}.mode = merge-on-read` table properties.\nThe construct rejects merge-on-read on a v1 table at synth time.\n\n### Can I create an Iceberg v3 table?\n\nYes. Set `formatVersion: IcebergFormatVersion.V3`; see [Format\nversion 3](#format-version-3) for the types, the engine matrix and the\nlimits.\n\n## Contributing\n\nDevelopment, the monorepo layout, and the demo app live in [CONTRIBUTING.md](https://github.com/ksco92/arceus/blob/main/CONTRIBUTING.md).\n\n## License\n\n[MIT](https://github.com/ksco92/arceus/blob/main/LICENSE).\n"
|
|
9754
|
+
"markdown": "# cdk-glue-iceberg-table\n\n[](https://github.com/ksco92/arceus/actions/workflows/ci.yml)\n[](https://codecov.io/gh/ksco92/arceus)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://pypi.org/project/cdk-glue-iceberg-table/)\n[](https://constructs.dev/packages/cdk-glue-iceberg-table)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://github.com/ksco92/arceus/blob/main/LICENSE)\n\n**`cdk-glue-iceberg-table` is an AWS CDK L2 construct to create and\nevolve Apache Iceberg tables in the AWS Glue Data Catalog — schema and\npartition changes that survive CloudFormation updates, via cdk\ndeploy.** It supports Iceberg format version 3 (`variant`, nanosecond\ntimestamps, `geometry` / `geography`, `unknown`). It emits the exact\n`AWS::Glue::Table` / `OpenTableFormatInput` shape, so a single `cdk\ndeploy` creates, evolves, and destroys a table like any other CDK\nresource. No custom resource, no Lambda, no \"`cdk deploy` and then run\nthis SQL by hand\" two-step. Table changes (new columns, renames, drops,\nnew partition fields) land as a reviewed diff in a pull request and\napply through `cdk deploy`.\n\nStatus (October 2026): published on npm (TypeScript/JavaScript) and\nPyPI (Python), pre-1.0, with a public surface guarded by an end-to-end\nconsumer test on every PR plus real-AWS integration suites, including\nones for format version 3. The multi-language packages are generated\nfrom one TypeScript source with [jsii](https://aws.github.io/jsii/)\nand indexed on [Construct\nHub](https://constructs.dev/packages/cdk-glue-iceberg-table).\nDeveloped in the [`ksco92/arceus`](https://github.com/ksco92/arceus)\nmonorepo, which also holds a CDK demo app that dogfoods the construct\nagainst a real AWS account.\n\n### Why this exists\n\nAWS documents the CloudFormation shape for Iceberg tables ([AWS Big\nData blog, December 2025](https://aws.amazon.com/blogs/big-data/create-and-update-apache-iceberg-tables-with-partitions-in-the-aws-glue-data-catalog-using-the-aws-sdk-and-aws-cloudformation)),\nbut raw `AWS::Glue::Table` is a minefield: the CREATE succeeds and the\n*first* `Update` silently strips `table_type=ICEBERG`, after which\nevery Athena query fails with `HIVE_UNSUPPORTED_FORMAT`. The\nmotivating CDK issue is [aws/aws-cdk#29660](https://github.com/aws/aws-cdk/issues/29660);\n[`manmartgarc`'s comment](https://github.com/aws/aws-cdk/issues/29660#issuecomment-4411359248)\ndocuments the only working shape and the silent-corruption traps you\nhit by getting it slightly wrong. This construct implements that\nshape, refuses to emit the unsafe alternatives, and pins Iceberg field\nIDs (declared [in creation order](#using-icebergtable) for a new\ntable) so schema evolution never orphans existing data. It was the\nbasis of an upstream proposal to land an `IcebergTable` in\n`@aws-cdk/aws-glue-alpha` ([aws/aws-cdk#37988](https://github.com/aws/aws-cdk/pull/37988)),\nclosed unmerged in September 2026.\n\n### How it compares\n\n| Approach | Declarative IaC (in the PR) | In-place schema + partition evolution | Prevents the silent-corruption footguns | Typed API + synth-time validation |\n| --- | --- | --- | --- | --- |\n| Raw `CfnTable` (L1) | ✅ | ⚠️ only if you hand-write the exact `OpenTableFormatInput` shape | ❌ you own both footguns | ❌ |\n| CDK custom resource (Lambda + Glue SDK) | ⚠️ via a custom resource you maintain | ⚠️ you write the diff logic | ⚠️ your code's responsibility | ❌ |\n| Spark SQL DDL, or Athena SQL DDL, at job runtime | ❌ imperative, outside the PR | ✅ but outside CloudFormation | n/a | ❌ |\n| **`cdk-glue-iceberg-table`** | ✅ | ✅ via `cdk deploy` | ✅ both prevented by construction | ✅ |\n\n## Install\n\nTypeScript / JavaScript (npm):\n\n```bash\nnpm install cdk-glue-iceberg-table\n```\n\nPython (PyPI):\n\n```bash\npip install cdk-glue-iceberg-table\n```\n\nPeer dependencies (your CDK app must already have these). This\nrelease requires `aws-cdk-lib` 2.273.0 or newer and `constructs`\n10.8.1 or newer, plus Node.js 22.14 or newer for both languages (the\nPython package runs under Node.js through jsii).\n\nFor TypeScript / JavaScript:\n\n```bash\nnpm install aws-cdk-lib@^2.273.0 constructs@^10.8.1\n```\n\nFor Python:\n\n```bash\npip install 'aws-cdk-lib>=2.273.0' 'constructs>=10.8.1'\n```\n\n## Use\n\n### TypeScript\n\n```ts\nimport { Bucket } from 'aws-cdk-lib/aws-s3';\nimport { Database } from 'aws-cdk-lib/aws-glue';\nimport {\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n IcebergPartitionTransform,\n} from 'cdk-glue-iceberg-table';\n\nconst bucket = new Bucket(this, 'Warehouse');\nconst db = new Database(this, 'Db', { databaseName: 'analytics' });\n\nnew IcebergTable(this, 'OrdersTable', {\n database: db,\n tableName: 'orders',\n location: `s3://${bucket.bucketName}/analytics/orders/`,\n columns: [\n { name: 'order_id', type: IcebergType.LONG, required: true, id: 1 },\n { name: 'customer_id', type: IcebergType.LONG, required: true, id: 2 },\n { name: 'placed_at', type: IcebergType.TIMESTAMPTZ, required: true, id: 3 },\n ],\n partitionSpec: [\n { sourceColumn: 'placed_at', transform: IcebergPartitionTransform.DAY, fieldId: 1000 },\n { sourceColumn: 'customer_id', transform: IcebergPartitionTransform.bucket(16), fieldId: 1001 },\n ],\n identifierFieldNames: ['order_id'],\n formatVersion: IcebergFormatVersion.V2,\n});\n```\n\n### Python\n\nThe same table in Python — the API mirrors TypeScript with\n`snake_case` props and `PascalCase` types:\n\n```python\nfrom aws_cdk.aws_s3 import Bucket\nfrom aws_cdk.aws_glue import Database\nfrom cdk_glue_iceberg_table import (\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n IcebergPartitionTransform,\n)\n\nbucket = Bucket(self, \"Warehouse\")\ndb = Database(self, \"Db\", database_name=\"analytics\")\n\nIcebergTable(self, \"OrdersTable\",\n database=db,\n table_name=\"orders\",\n location=f\"s3://{bucket.bucket_name}/analytics/orders/\",\n columns=[\n {\"name\": \"order_id\", \"type\": IcebergType.LONG, \"required\": True, \"id\": 1},\n {\"name\": \"customer_id\", \"type\": IcebergType.LONG, \"required\": True, \"id\": 2},\n {\"name\": \"placed_at\", \"type\": IcebergType.TIMESTAMPTZ, \"required\": True, \"id\": 3},\n ],\n partition_spec=[\n {\"source_column\": \"placed_at\", \"transform\": IcebergPartitionTransform.DAY, \"field_id\": 1000},\n {\"source_column\": \"customer_id\", \"transform\": IcebergPartitionTransform.bucket(16), \"field_id\": 1001},\n ],\n identifier_field_names=[\"order_id\"],\n format_version=IcebergFormatVersion.V2,\n)\n```\n\nConsumer-facing reference sections:\n\n- [How it compares](#how-it-compares) — `cdk-glue-iceberg-table` vs raw `CfnTable`, custom resources, and runtime SQL DDL.\n- [Using `IcebergTable`](#using-icebergtable) — full API reference with examples.\n- [Format version 3](#format-version-3) — the six v3 types, which engines read them, and the accepted limits.\n- [Two footguns the construct prevents](#two-footguns-the-construct-prevents) — the silent-corruption traps that motivated this construct.\n- [Known limitations](#known-limitations) — what the construct does and doesn't enforce.\n- [FAQ](#faq) — common \"how do I … in CDK / CloudFormation\" questions.\n\n## Exported surface\n\nThe package entry point re-exports the main types you import from\n`cdk-glue-iceberg-table`. In TypeScript / JavaScript the compiled\nsurface lives under `dist/lib/iceberg/` after `npm install`; in Python\nthe same surface is imported from the `cdk_glue_iceberg_table` module\nafter `pip install`:\n\n- **`IcebergTable`** — the L2 construct itself, with `grantRead` / `grantWrite` / `grantReadWrite`, the `fromIcebergTableAttributes(...)` import factory, and `creationOrderFieldIds(columns)` (Python: `creation_order_field_ids`), which returns a map from field path to the id Glue gives that field in a new table; a name containing `.` or a backquote appears in backquotes.\n- **`IcebergType`** — primitive types, the v3 constants `UNKNOWN`, `TIMESTAMP_NS`, `TIMESTAMPTZ_NS`, and `VARIANT`, and the `list` / `map` / `struct` / `decimal` / `fixed` / `geometry(crs?)` / `geography(crs?, algorithm?)` factories. `list` takes an `IcebergListDefinition`, `map` an `IcebergMapDefinition`, and `struct` a list of `IcebergStructFieldDefinition`, each carrying the nested field ids. Renders to the string Glue's `IcebergStructField.type` expects.\n- **`IcebergPartitionTransform`** — identity / `bucket(N)` / `truncate(W)` / year / month / day / hour / void. Each transform validates against the source column type at synth time.\n- **`IcebergDataFormat`** (parquet/orc/avro, default parquet), **`IcebergFormatVersion`** (v1/v2/v3, required — set explicitly per table; Athena cannot query v3 tables), **`IcebergSortDirection`**, **`IcebergNullOrder`**, **`IcebergEdgeAlgorithm`** (`SPHERICAL`, `VINCENTY`, `THOMAS`, `ANDOYER`, `KARNEY`; the second argument of `IcebergType.geography`), plus a `tableProperties` validator that catches misconfigured properties before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).\n\n## Using `IcebergTable`\n\nA minimal table:\n\n```typescript\nimport {\n Bucket,\n} from 'aws-cdk-lib/aws-s3';\nimport {\n Database,\n} from 'aws-cdk-lib/aws-glue';\nimport {\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nconst bucket = new Bucket(this, 'Warehouse');\n\nconst db = new Database(this, 'Db', {\n databaseName: 'analytics',\n});\n\nnew IcebergTable(this, 'Users', {\n database: db,\n tableName: 'users',\n columns: [\n {\n name: 'user_id',\n type: IcebergType.LONG,\n required: true,\n id: 1,\n },\n {\n name: 'email',\n type: IcebergType.STRING,\n required: true,\n id: 2,\n },\n {\n name: 'signed_up_at',\n type: IcebergType.TIMESTAMPTZ,\n required: true,\n id: 3,\n },\n ],\n location: `s3://${bucket.bucketName}/analytics/users/`,\n formatVersion: IcebergFormatVersion.V2,\n});\n```\n\nEvery top-level column must declare an `id`: a stable, unique,\npositive integer you pick once and never reuse or reassign. Because\nIceberg data files reference fields by id rather than by name, a fixed\nid, declared in creation order when the table is created (below), is\nwhat lets you add, rename, and drop columns across deploys without\ncorrupting old data — see\n[How do I evolve an Iceberg table schema](#how-do-i-evolve-an-iceberg-table-schema-add-rename-or-drop-a-column-in-cloudformation).\nOnce a column is dropped, leave its id retired; never hand it to a new\ncolumn. Nested fields follow the same rule: every struct field takes\nan `id`, every list an `elementId`, and every map a `keyId` and a\n`valueId`. All ids in a schema, top-level and nested, must be distinct\nintegers from 1 to 2147483447.\n\nA new table must declare its ids in creation order. When Glue creates\na table it does not keep the declared ids: it numbers the fields from\n1 in creation order and points the partition spec, sort order and\nidentifier fields at the new ids. An update Glue accepts stores the\ndeclared ids as given, so the first such update of a table created\nwith other ids renumbers its fields; for the update Glue rejects, see\n[Known limitations](#known-limitations). To number a new table, number\nthe top-level columns first, in order, then the nested fields of each\ncolumn in turn (a list's element; a map's key, then its value, then\nthe key's nested fields, then the value's; a struct's fields, then\ntheir nested fields). `IcebergTable.creationOrderFieldIds` returns\nthese ids by field path and ignores the ids in its input:\n\n```typescript\nconst columns = [\n { name: 'id', type: IcebergType.LONG, required: true, id: 1 },\n { name: 'tags', type: IcebergType.list({ element: IcebergType.STRING, elementId: 4 }), id: 2 },\n {\n name: 'attrs',\n type: IcebergType.map({ key: IcebergType.STRING, keyId: 5, value: IcebergType.STRING, valueId: 6 }),\n id: 3,\n },\n];\nIcebergTable.creationOrderFieldIds(columns);\n// { id: 1, tags: 2, attrs: 3, 'tags.element': 4, 'attrs.key': 5, 'attrs.value': 6 }\n```\n\nIds out of creation order are for fields a later deploy adds; see\n[Evolving schema and partitions](#evolving-schema-and-partitions).\n\nA table that exercises most of the surface (partitions, sort order,\nnested types, identifier fields, table properties, removal policy):\n\n```typescript\nimport {\n RemovalPolicy,\n} from 'aws-cdk-lib';\nimport {\n Database,\n} from 'aws-cdk-lib/aws-glue';\nimport {\n IcebergDataFormat,\n IcebergFormatVersion,\n IcebergNullOrder,\n IcebergPartitionTransform,\n IcebergSortDirection,\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nnew IcebergTable(this, 'OrdersTable', {\n database: db,\n tableName: 'orders',\n comment: 'Demo Iceberg orders table — exercises partitions, sort order, and merge-on-read.',\n columns: [\n {\n name: 'order_id',\n type: IcebergType.LONG,\n required: true,\n id: 1,\n },\n {\n name: 'customer_id',\n type: IcebergType.LONG,\n required: true,\n id: 2,\n },\n {\n name: 'order_amount',\n type: IcebergType.decimal(12, 2),\n required: true,\n id: 3,\n },\n {\n name: 'currency',\n type: IcebergType.STRING,\n required: true,\n id: 4,\n },\n {\n name: 'placed_at',\n type: IcebergType.TIMESTAMPTZ,\n required: true,\n id: 5,\n },\n {\n name: 'tags',\n type: IcebergType.list({\n element: IcebergType.STRING,\n elementId: 9,\n }),\n id: 6,\n },\n {\n name: 'shipping_address',\n type: IcebergType.struct([\n {\n id: 10,\n name: 'line1',\n type: IcebergType.STRING,\n required: true,\n },\n {\n id: 11,\n name: 'city',\n type: IcebergType.STRING,\n required: true,\n },\n {\n id: 12,\n name: 'country',\n type: IcebergType.STRING,\n required: true,\n },\n {\n id: 13,\n name: 'postal_code',\n type: IcebergType.STRING,\n },\n ]),\n id: 7,\n },\n {\n name: 'metadata',\n type: IcebergType.map({\n key: IcebergType.STRING,\n keyId: 14,\n value: IcebergType.STRING,\n valueId: 15,\n }),\n id: 8,\n },\n ],\n location: `s3://${bucket.bucketName}/analytics/orders/`,\n partitionSpec: [\n {\n sourceColumn: 'placed_at',\n transform: IcebergPartitionTransform.DAY,\n fieldId: 1000,\n },\n {\n sourceColumn: 'customer_id',\n transform: IcebergPartitionTransform.bucket(16),\n fieldId: 1001,\n },\n ],\n sortOrder: [\n {\n sourceColumn: 'placed_at',\n direction: IcebergSortDirection.ASC,\n nullOrder: IcebergNullOrder.NULLS_LAST,\n },\n {\n sourceColumn: 'order_id',\n direction: IcebergSortDirection.ASC,\n },\n ],\n identifierFieldNames: [\n 'order_id',\n ],\n dataFormat: IcebergDataFormat.PARQUET,\n formatVersion: IcebergFormatVersion.V2,\n tableProperties: {\n 'write.parquet.compression-codec': 'zstd',\n 'write.delete.mode': 'merge-on-read',\n 'write.update.mode': 'merge-on-read',\n 'write.merge.mode': 'merge-on-read',\n 'write.target-file-size-bytes': '134217728',\n 'history.expire.min-snapshots-to-keep': '5',\n 'gc.enabled': 'true',\n },\n removalPolicy: RemovalPolicy.DESTROY,\n});\n```\n\nEach partition field also pins a `fieldId`: an integer of 1000 or\nabove (the Iceberg spec reserves lower ids for schema columns), unique\nacross the table's entire partition-spec history. The discipline is\nthe inverse of the column-id rule. A column keeps its id forever; a\npartition field keeps its id only while it stays untouched. Change\nanything about a field — swap `month` for `day`, point it at a\ndifferent source column — and it becomes a *new* field that needs a\nfresh, never-before-used id, while the old id retires with the old\nspec. Reusing a retired id (or reassigning an id to a different field)\nproduces a table that Athena still reads but Spark and Trino reject\nwith `ValidationException: Conflicting partition fields`.\n\n`identifierFieldNames` lists the columns that together identify a row\nand renders as the schema's `identifier-field-ids`. Each one must be a\n`required: true` column (Glue rejects an optional one), and `cdk synth`\nrejects an identifier column of type float, double, list, map, struct,\nunknown, variant, geometry or geography. The construct prefixes each\nidentifier column's `doc` with `(PK) `, so a `doc` you write must not\nstart with `(PK) `; see [Evolving schema and\npartitions](#evolving-schema-and-partitions).\n\nThe resulting Iceberg `metadata.json` for this table contains every\nfeature you set:\n\n```json\n{\n \"format-version\": 2,\n \"table-uuid\": \"39a948f9-...\",\n \"current-schema-id\": 0,\n \"schemas\": [\n {\n \"schema-id\": 0,\n \"identifier-field-ids\": [1],\n \"fields\": [\n { \"id\": 1, \"name\": \"order_id\", \"required\": true, \"type\": \"long\", \"doc\": \"(PK) \" },\n { \"id\": 2, \"name\": \"customer_id\", \"required\": true, \"type\": \"long\" },\n { \"id\": 3, \"name\": \"order_amount\", \"required\": true, \"type\": \"decimal(12, 2)\" },\n { \"id\": 4, \"name\": \"currency\", \"required\": true, \"type\": \"string\" },\n { \"id\": 5, \"name\": \"placed_at\", \"required\": true, \"type\": \"timestamptz\" },\n { \"id\": 6, \"name\": \"tags\", \"required\": false,\n \"type\": { \"type\": \"list\", \"element-id\": 9, \"element\": \"string\", \"element-required\": false } },\n { \"id\": 7, \"name\": \"shipping_address\", \"required\": false,\n \"type\": { \"type\": \"struct\", \"fields\": [\n { \"id\": 10, \"name\": \"line1\", \"required\": true, \"type\": \"string\" },\n { \"id\": 11, \"name\": \"city\", \"required\": true, \"type\": \"string\" },\n { \"id\": 12, \"name\": \"country\", \"required\": true, \"type\": \"string\" },\n { \"id\": 13, \"name\": \"postal_code\", \"required\": false, \"type\": \"string\" }\n ] } },\n { \"id\": 8, \"name\": \"metadata\", \"required\": false,\n \"type\": { \"type\": \"map\", \"key-id\": 14, \"key\": \"string\", \"value-id\": 15,\n \"value-required\": false, \"value\": \"string\" } }\n ]\n }\n ],\n \"partition-specs\": [\n { \"spec-id\": 0, \"fields\": [\n { \"name\": \"placed_at_day\", \"transform\": \"day\", \"source-id\": 5, \"field-id\": 1000 },\n { \"name\": \"customer_id_bucket\", \"transform\": \"bucket[16]\", \"source-id\": 2, \"field-id\": 1001 }\n ]}\n ],\n \"sort-orders\": [\n { \"order-id\": 1, \"fields\": [\n { \"transform\": \"identity\", \"source-id\": 5, \"direction\": \"asc\", \"null-order\": \"nulls-last\" },\n { \"transform\": \"identity\", \"source-id\": 1, \"direction\": \"asc\", \"null-order\": \"nulls-last\" }\n ]}\n ],\n \"properties\": {\n \"format-version\": \"2\",\n \"write.format.default\": \"parquet\",\n \"write.parquet.compression-codec\": \"zstd\",\n \"write.merge.mode\": \"merge-on-read\",\n \"write.update.mode\": \"merge-on-read\",\n \"write.delete.mode\": \"merge-on-read\",\n \"write.target-file-size-bytes\": \"134217728\",\n \"history.expire.min-snapshots-to-keep\": \"5\",\n \"gc.enabled\": \"true\",\n \"comment\": \"Demo Iceberg orders table — exercises partitions, sort order, and merge-on-read.\"\n }\n}\n```\n\n### Format version 3\n\nSet `formatVersion: IcebergFormatVersion.V3` to create a v3 table. The\nsix v3 types (below) are only accepted on a V3 table, at any nesting\ndepth: inside a struct, as a list element, or as a map key or value.\nSynth fails on a V1 or V2 table that uses one.\n\n```typescript\nimport {\n IcebergEdgeAlgorithm,\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nnew IcebergTable(this, 'Events', {\n database: db,\n tableName: 'events',\n location: `s3://${bucket.bucketName}/analytics/events/`,\n columns: [\n { name: 'occurred_at', type: IcebergType.TIMESTAMP_NS, id: 1 },\n { name: 'payload', type: IcebergType.VARIANT, id: 2 },\n {\n name: 'origin',\n type: IcebergType.geography('OGC:CRS84', IcebergEdgeAlgorithm.SPHERICAL),\n id: 3,\n },\n ],\n formatVersion: IcebergFormatVersion.V3,\n});\n```\n\nThe same table in Python:\n\n```python\nfrom cdk_glue_iceberg_table import (\n IcebergEdgeAlgorithm,\n IcebergFormatVersion,\n IcebergTable,\n IcebergType,\n)\n\nIcebergTable(self, \"Events\",\n database=db,\n table_name=\"events\",\n location=f\"s3://{bucket.bucket_name}/analytics/events/\",\n columns=[\n {\"name\": \"occurred_at\", \"type\": IcebergType.TIMESTAMP_NS, \"id\": 1},\n {\"name\": \"payload\", \"type\": IcebergType.VARIANT, \"id\": 2},\n {\n \"name\": \"origin\",\n \"type\": IcebergType.geography(\"OGC:CRS84\", IcebergEdgeAlgorithm.SPHERICAL),\n \"id\": 3,\n },\n ],\n format_version=IcebergFormatVersion.V3,\n)\n```\n\nThis table has a `timestamp_ns` column; see [Known limitations](#known-limitations)\nbefore reading it from upstream Spark.\n\nThe v3 types, with the rules `cdk synth` enforces:\n\n| Type | Factory | Rendered string | Partition and identifier rules |\n| --- | --- | --- | --- |\n| unknown | `IcebergType.UNKNOWN` | `unknown` | Must be optional, as a column, list element, or map value; never a map key. Partition source only with `void`. Not an identifier field. |\n| timestamp with nanoseconds | `IcebergType.TIMESTAMP_NS` | `timestamp_ns` | `identity`, `year`, `month`, `day`, `hour`, `bucket(N)`, `void`; not `truncate(W)`. Allowed as an identifier field. |\n| timestamptz with nanoseconds | `IcebergType.TIMESTAMPTZ_NS` | `timestamptz_ns` | Same as `timestamp_ns`. |\n| variant | `IcebergType.VARIANT` | `variant` | Never a partition or sort source, `void` included. Not an identifier field. |\n| geometry | `IcebergType.geometry(crs?)` | `geometry`, or `geometry(<crs>)` when a CRS is given | Partition source only with `void`. Not an identifier field. |\n| geography | `IcebergType.geography(crs?, algorithm?)` | `geography`, `geography(<crs>)`, or `geography(<crs>, <algorithm>)` when an algorithm is given, such as `geography(OGC:CRS84, spherical)` | Same as `geometry`. |\n\nA geometry or geography `crs` is an identifier such as `OGC:CRS84` or\n`EPSG:3857`; whitespace, parentheses, braces, and commas are rejected.\nWhen `algorithm` is set without a `crs`, the rendered string spells out\n`OGC:CRS84`.\n\nWhich engines read a v3 table created by the construct:\n\n| Engine | v3 tables | Notes |\n| --- | --- | --- |\n| Spark 4.1 + Iceberg 1.12 (the integration-test runner) | Yes | Reads and writes tables the construct creates, including deletion vectors, row lineage, `variant`, `unknown`, and `geometry` / `geography`, except tables with nanosecond-timestamp columns or a geometry CRS other than `OGC:CRS84` (checked by `scripts/spark/verify_v3_dml.py`; details in [Known limitations](#known-limitations)). |\n| AWS Glue 6.0 | Documented | The [AWS Glue versions](https://docs.aws.amazon.com/glue/latest/dg/release-notes.html) page lists \"Iceberg format version 3 with VARIANT data type, deletion vectors, row lineage tracking, nanosecond timestamps, and geo types\". The integration suites do not run on Glue 6.0. |\n| Amazon EMR Spark 8.1.0 (Spark 4.1.1, Iceberg 1.11.0-amzn-0) | Documented | The [emr-spark-8.1.0 release notes](https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-spark810-release.html) state \"Amazon EMR Spark 8.1.0 extends Iceberg v3 spec support for Spark workloads\". The integration suites do not run on EMR 8.1. |\n| Athena SQL | No | The [Athena Iceberg page](https://docs.aws.amazon.com/athena/latest/ug/querying-iceberg.html) states \"Athena only creates and operates on Iceberg v2 tables.\" |\n\n### Granting access\n\n```typescript\ntable.grantRead(role); // Glue read + S3 read on the table's prefix\ntable.grantWrite(role); // Glue write + S3 write\ntable.grantReadWrite(role);\n```\n\nThe `grant*` helpers issue **IAM grants only**. Under Lake Formation\nyou still add the matching `SELECT` / `INSERT` / `DELETE` LF grants on\ntop of the construct's IAM grants for Athena queries to succeed.\n\n### Importing an existing table\n\n```typescript\nconst existing = IcebergTable.fromIcebergTableAttributes(this, 'Orders', {\n database: db,\n tableName: 'orders',\n location: 's3://my-bucket/analytics/orders/',\n});\nexisting.grantRead(role);\n```\n\n### Evolving schema and partitions\n\nChange the `columns` array (or `partitionSpec`) and run `cdk deploy`\nagain. The construct passes the new schema to Glue's `UpdateTable`,\nwhich writes a new `metadata.json` with a new `schema-id`; existing\ndata files stay readable because each column's `id` is pinned to the\nid the table holds and never reused. Adds, renames (same `id`, new\n`name`), and drops all flow through `cdk deploy` alone — no\nout-of-band SQL DDL.\n\nNested fields evolve the same way. Each struct field, list element,\nmap key, and map value keeps the id you gave it (the rule is under\n[Using `IcebergTable`](#using-icebergtable)), so adding a column or a\nstruct field anywhere in the schema leaves every other id unchanged. A\nnew nested field takes an id no field of the table has ever used:\n\n```typescript\ntype: IcebergType.struct([\n { id: 10, name: 'line1', type: IcebergType.STRING },\n { id: 16, name: 'line2', type: IcebergType.STRING }, // new field, never-used id\n { id: 11, name: 'city', type: IcebergType.STRING },\n]),\n```\n\nSuch an id is out of creation order, which is expected once a table\nhas evolved: creation order applies only to the ids a table is created\nwith (see [Using `IcebergTable`](#using-icebergtable)).\n\nUpgrading from a version before 0.12.0, which numbered nested fields\nitself: declare the ids the table's current `metadata.json` holds,\ntop-level and nested (each field `id`, `element-id`, `key-id` and\n`value-id`), or the next deploy renumbers the fields that differ and\nexisting data files read under the wrong field. A nested type object\nshared by two columns now fails synth with a duplicate id; give each\ncolumn its own nested type with its own ids. The list element and map\nvalue nullability default also changed: `elementRequired` and\n`valueRequired` now default to `false`. Earlier versions made list\nelements required by default and map values required unless the third\nargument of `map` was `false`; set `elementRequired: true` or\n`valueRequired: true` wherever the deployed schema has\n`\"element-required\": true` or `\"value-required\": true`.\n\nGlue's `UpdateTable` reuses an earlier schema whose fields (name,\ntype, `required`, `doc`) match the request, together with that\nschema's `identifier-field-ids`. The construct therefore prefixes the\n`doc` of every identifier column with `(PK) ` (just `(PK) ` when the\ncolumn has no `doc`), shown as the column comment, so every\nidentifier change is a field change. On a table whose schema history\nwas written with the marker, identifier changes flow through `cdk\ndeploy` like column changes.\n\nPartition evolution flows the same way, with one extra rule on\n`fieldId`: a changed partition field is a new partition field. To move\na table from `month(placed_at)` to `day(placed_at)`, drop the month\nentry and add a day entry under a fresh `fieldId` — never recycle the\nmonth field's id:\n\n```typescript\npartitionSpec: [\n // was: { sourceColumn: 'placed_at', transform: IcebergPartitionTransform.MONTH, fieldId: 1000 },\n {\n sourceColumn: 'placed_at',\n transform: IcebergPartitionTransform.DAY,\n fieldId: 1001,\n },\n],\n```\n\nOld data files written under the month spec stay readable; new writes\npartition by day. Reusing `1000` for the day field instead corrupts\nthe spec history for strict engines (Spark, Trino) while Athena keeps\nworking — the failure only surfaces at Spark runtime.\n\n### Inserting and querying\n\n```sql\n-- INSERT into the orders table\nINSERT INTO sample_database.orders VALUES\n (1001, 5001, DECIMAL '149.99', 'USD',\n TIMESTAMP '2026-05-20 09:15:00 UTC',\n ARRAY['holiday-promo','first-order'],\n CAST(ROW('1 Infinite Loop','Cupertino','US','95014')\n AS ROW(line1 VARCHAR,city VARCHAR,country VARCHAR,postal_code VARCHAR)),\n MAP(ARRAY['channel','utm'], ARRAY['web','google'])),\n -- ... more rows\n;\n\n-- merge-on-read DELETE (only legal because we chose v2 + merge-on-read mode)\nDELETE FROM sample_database.orders WHERE order_id = 1003;\n\n-- merge-on-read UPDATE\nUPDATE sample_database.orders SET currency = 'GBP' WHERE customer_id = 5002;\n\n-- SELECT\nSELECT customer_id, SUM(order_amount) AS total\n FROM sample_database.orders\n GROUP BY 1\n ORDER BY 2 DESC;\n```\n\n## Two footguns the construct prevents\n\n### Footgun #1 — schema under `storageDescriptor.columns`\n\nThe CREATE succeeds but the first UPDATE silently strips\n`table_type=ICEBERG` from the table's Glue parameters, and Athena\nqueries after that fail with `HIVE_UNSUPPORTED_FORMAT`.\n\n```typescript\n// DON'T DO THIS — what most StackOverflow / re:Post examples show\nnew CfnTable(this, 'OrdersBad', {\n catalogId: this.account,\n databaseName: 'analytics',\n tableInput: {\n name: 'orders',\n tableType: 'EXTERNAL_TABLE',\n parameters: {\n table_type: 'ICEBERG',\n },\n storageDescriptor: {\n location: 's3://.../orders/',\n columns: [\n /* ... */\n ],\n },\n },\n openTableFormatInput: {\n icebergInput: {\n metadataOperation: 'CREATE',\n version: '2',\n },\n },\n});\n```\n\n`IcebergTable` instead always emits schema/partitions/sort/properties\nunder `openTableFormatInput.icebergInput.icebergTableInput`, never\nunder `storageDescriptor`.\n\n### Footgun #2 — `tableInput` co-present with `openTableFormatInput`\n\nEven setting just `tableInput: { name: 'foo' }` next to\n`openTableFormatInput` returns\n`\"Table metadata is expected only via TableInput or via IcebergTableInputProperties inside OpenTableFormatInput\"`.\nThe construct never emits `tableInput`; the table-level comment goes\ninto `tableProperties['comment']`, which lives inside\n`icebergTableInput.properties`.\n\n(There is a third footgun, field-id reuse after a column drop, that\nthe construct does **not** prevent. See the next section.)\n\n## Known limitations\n\n- **Creation order is not checked.** The construct cannot tell at synth whether a table is new or already deployed, so it accepts a new table whose ids are not in creation order (the rule is under [Using `IcebergTable`](#using-icebergtable)). When the first update of such a table changes the id of a column an identity partition is sourced from, Glue rejects the update with `Cannot create identity partition sourced from different field in schema` and the CloudFormation rollback fails the same way, leaving the stack in `UPDATE_ROLLBACK_FAILED`.\n- **Field-id reuse is not detected across deploys.** This applies to nested ids (struct field, list element, map key and value) as well as column ids. If you drop a column with `id = 5` and then add a different column with `id = 5` in a later deploy, Glue accepts the UPDATE and Iceberg's metadata silently violates the \"never reuse a retired id\" invariant. Readers projecting old snapshots will surface deleted data under the new field's name. The construct enforces uniqueness **within one deploy** (`duplicate field id N` validator), but it doesn't compare against the live table state. The safe workflow is to treat dropped ids as retired forever.\n- **Partition field-id reuse is not detected across deploys either.** The same live-state blindness applies to partition `fieldId`s: the construct validates them within one deploy (integer >= 1000, unique in the spec), but it cannot tell that the id you pinned already appeared in an earlier spec of the deployed table. The invariant partition ids must uphold is stricter than the column one — unique across the table's **entire spec history**, so a changed field (different transform or source column) must take a fresh id rather than keep its old one. Violate it and strict engines (Spark, Trino) reject the table with `Conflicting partition fields` while Athena keeps answering, masking the break until Spark runtime. The safe workflow: treat every id that has ever appeared in any spec as retired, and allocate new fields the next never-used id. A partition change needs a new source column or transform; Glue ignores a `fieldId`-only change.\n- **`last-partition-id` is written by Glue, not by the construct.** The construct supplies the spec's field ids; Glue's `UpdateTable` computes the table's `last-partition-id` counter when it writes the new metadata. The repo's integration test asserts the counter keeps up with the highest supplied field id after a partition evolution, but engines that later evolve the spec themselves (e.g. Spark `ALTER TABLE … ADD PARTITION FIELD`) allocate from that counter — if you mix CDK-managed and engine-managed partition evolution, verify the counter in `metadata.json` before trusting engine-side changes.\n- **Removing the last identifier from a table created with identifiers before 0.11.0 can bring the old ids back.** That table's history holds an unmarked schema with ids, and Glue reuses it when the rendered fields match it again. Change a column `doc` in the same deploy that removes the last identifier.\n- **CREATE-only metadata operation.** The CFN `IcebergInput.metadataOperation` only accepts `CREATE`; the construct always emits that. Subsequent deploys use Glue's normal `UpdateTable` path, which writes new Iceberg metadata in-place.\n- **Format version is immutable after CREATE.** Glue ignores a V2 to V3 `formatVersion` change, though `properties.format-version` then reads `3`. Keep `formatVersion` at the table's real version, or destroy + recreate the table to change it. An engine can upgrade a v2 table to v3 in place with `ALTER TABLE ... SET TBLPROPERTIES ('format-version' = '3')`; after that, set `formatVersion` to `V3` so the next deploy declares the version the table already has.\n- **`merge-on-read` requires v2 or later.** The construct rejects `write.{delete,update,merge}.mode = merge-on-read` on a v1 table at synth time.\n- **Column defaults are not available.** Iceberg v3 `initial-default` and `write-default` cannot be declared through CloudFormation: the `AWS::Glue::Table` resource schema has no `InitialDefault` or `WriteDefault` property. Revisit when that schema gains the two properties.\n- **Nanosecond timestamps are catalog-only for upstream Spark 4.1.** A table with any `timestamp_ns` or `timestamptz_ns` column fails to open in upstream Spark 4.1 with Iceberg 1.12. The integration suite for these columns asserts from `metadata.json` only. The [engine matrix](#format-version-3) lists the engines that document support.\n- **Geometry CRS other than `OGC:CRS84` is catalog-only for upstream Spark 4.1.** Spark 4.1 accepts `OGC:CRS84`, or bare `geometry`, as the geometry CRS; a table declaring another CRS, such as `EPSG:3857`, cannot be read there.\n- **Athena DDL features that don't surface through CFN** (e.g. `ALTER TABLE WRITE ORDERED BY`, `ALTER TABLE … SET LOCATION`, `bucketed_by` / `bucket_count` Hive clauses) are not exposed. Use `IcebergPartitionTransform.bucket(N)` instead of Hive bucketing.\n- **Dropping a partition column requires a `void` intermediate per the Iceberg spec**, and the CFN `OpenTableFormatInput` cannot express that. The construct accepts the change, but Athena queries against the result will fail with `Type cannot be null`. The safe pattern is to drop partitions that source from a column while keeping that column in the schema, and only drop a column once it is no longer partitioning anything.\n\n## FAQ\n\n### How do I create an Apache Iceberg table in AWS CDK?\n\nInstall `cdk-glue-iceberg-table`, then declare an `IcebergTable` with a\ndatabase, a column list, and an S3 `location` — see [Use](#use).\n`cdk deploy` creates the Glue Data Catalog table and writes the\nIceberg `metadata.json` under your S3 prefix. No custom resource or\nLambda is involved; it is a plain `AWS::Glue::Table` in your\nCloudFormation template.\n\n### How do I evolve an Iceberg table schema (add, rename, or drop a column) in CloudFormation?\n\nChange the `columns` array and run `cdk deploy` again. The construct\npasses the new schema to Glue's `UpdateTable`, which writes a new\n`metadata.json` with a new `schema-id`; existing data files stay\nreadable because each column's `id` is pinned to the id the table\nholds and never reused.\nPartition changes work the same way, except a changed partition field\ntakes a fresh `fieldId` instead of keeping its old one. See [Evolving\nschema and partitions](#evolving-schema-and-partitions).\n\n### Does CloudFormation support Iceberg tables natively?\n\nYes, through `AWS::Glue::Table` with `OpenTableFormatInput.IcebergInput`,\ndocumented by AWS in [December 2025](https://aws.amazon.com/blogs/big-data/create-and-update-apache-iceberg-tables-with-partitions-in-the-aws-glue-data-catalog-using-the-aws-sdk-and-aws-cloudformation).\nThe catch is that the raw shape corrupts the table on the first\n`Update` if you place schema under `storageDescriptor.columns` or set\n`tableInput` alongside `openTableFormatInput`. This construct only ever\nemits the safe shape and gives you no way to express the unsafe ones.\nSee [Two footguns the construct prevents](#two-footguns-the-construct-prevents).\n\n### What is the difference between cdk-glue-iceberg-table and a raw CfnTable?\n\nA raw `CfnTable` makes you hand-write the `OpenTableFormatInput` JSON\nand own both silent-corruption footguns. `cdk-glue-iceberg-table` gives\nyou a typed `IcebergType` / `IcebergPartitionTransform` API,\nsynth-time validation (partition transforms checked against column\ntypes, format/version mismatches caught before deploy), pinned field\nIDs for safe evolution (declared in creation order for a new table),\nand `grantRead` / `grantWrite` helpers. See\n[How it compares](#how-it-compares).\n\n### How do I create a partitioned Iceberg table in CDK?\n\nPass a `partitionSpec` of `IcebergPartitionTransform` entries —\n`identity`, `bucket(N)`, `truncate(W)`, `year`, `month`, `day`, `hour`,\nor `void` — each pinned to a `fieldId` of 1000 or above. Transforms are\nvalidated against their source column's type at synth time. See\n[Using `IcebergTable`](#using-icebergtable).\n\n### Does it work with Athena and Lake Formation?\n\nYes for v2 tables, no for v3 (see the engine matrix in [Format\nversion 3](#format-version-3)). On v2, Athena reads and writes,\nincluding merge-on-read `INSERT` / `UPDATE` / `DELETE` / `MERGE`, time\ntravel, `OPTIMIZE`, and `VACUUM`. The demo app in the\n[`arceus`](https://github.com/ksco92/arceus) repo registers its tables\nwith Lake Formation and exercises the same operations through Spark in\nits integration suite. The construct's `grant*` helpers issue IAM\ngrants; under Lake Formation you still add the matching `SELECT` /\n`INSERT` LF grants. See [Granting access](#granting-access).\n\n### Can I use Iceberg v2 merge-on-read (row-level UPDATE and DELETE)?\n\nYes — set `formatVersion: IcebergFormatVersion.V2` and the\n`write.{delete,update,merge}.mode = merge-on-read` table properties.\nThe construct rejects merge-on-read on a v1 table at synth time.\n\n### Can I create an Iceberg v3 table?\n\nYes. Set `formatVersion: IcebergFormatVersion.V3`; see [Format\nversion 3](#format-version-3) for the types, the engine matrix and the\nlimits.\n\n## Contributing\n\nDevelopment, the monorepo layout, and the demo app live in [CONTRIBUTING.md](https://github.com/ksco92/arceus/blob/main/CONTRIBUTING.md).\n\n## License\n\n[MIT](https://github.com/ksco92/arceus/blob/main/LICENSE).\n"
|
|
9755
9755
|
},
|
|
9756
9756
|
"repository": {
|
|
9757
9757
|
"directory": "packages/cdk-glue-iceberg-table",
|
|
@@ -9779,7 +9779,7 @@
|
|
|
9779
9779
|
"kind": "interface",
|
|
9780
9780
|
"locationInModule": {
|
|
9781
9781
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9782
|
-
"line":
|
|
9782
|
+
"line": 219
|
|
9783
9783
|
},
|
|
9784
9784
|
"methods": [
|
|
9785
9785
|
{
|
|
@@ -9790,7 +9790,7 @@
|
|
|
9790
9790
|
},
|
|
9791
9791
|
"locationInModule": {
|
|
9792
9792
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9793
|
-
"line":
|
|
9793
|
+
"line": 233
|
|
9794
9794
|
},
|
|
9795
9795
|
"name": "grantRead",
|
|
9796
9796
|
"parameters": [
|
|
@@ -9815,7 +9815,7 @@
|
|
|
9815
9815
|
},
|
|
9816
9816
|
"locationInModule": {
|
|
9817
9817
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9818
|
-
"line":
|
|
9818
|
+
"line": 239
|
|
9819
9819
|
},
|
|
9820
9820
|
"name": "grantReadWrite",
|
|
9821
9821
|
"parameters": [
|
|
@@ -9840,7 +9840,7 @@
|
|
|
9840
9840
|
},
|
|
9841
9841
|
"locationInModule": {
|
|
9842
9842
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9843
|
-
"line":
|
|
9843
|
+
"line": 236
|
|
9844
9844
|
},
|
|
9845
9845
|
"name": "grantWrite",
|
|
9846
9846
|
"parameters": [
|
|
@@ -9869,7 +9869,7 @@
|
|
|
9869
9869
|
"immutable": true,
|
|
9870
9870
|
"locationInModule": {
|
|
9871
9871
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9872
|
-
"line":
|
|
9872
|
+
"line": 227
|
|
9873
9873
|
},
|
|
9874
9874
|
"name": "database",
|
|
9875
9875
|
"type": {
|
|
@@ -9885,7 +9885,7 @@
|
|
|
9885
9885
|
"immutable": true,
|
|
9886
9886
|
"locationInModule": {
|
|
9887
9887
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9888
|
-
"line":
|
|
9888
|
+
"line": 230
|
|
9889
9889
|
},
|
|
9890
9890
|
"name": "location",
|
|
9891
9891
|
"type": {
|
|
@@ -9901,7 +9901,7 @@
|
|
|
9901
9901
|
"immutable": true,
|
|
9902
9902
|
"locationInModule": {
|
|
9903
9903
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9904
|
-
"line":
|
|
9904
|
+
"line": 221
|
|
9905
9905
|
},
|
|
9906
9906
|
"name": "tableArn",
|
|
9907
9907
|
"type": {
|
|
@@ -9917,7 +9917,7 @@
|
|
|
9917
9917
|
"immutable": true,
|
|
9918
9918
|
"locationInModule": {
|
|
9919
9919
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9920
|
-
"line":
|
|
9920
|
+
"line": 224
|
|
9921
9921
|
},
|
|
9922
9922
|
"name": "tableName",
|
|
9923
9923
|
"type": {
|
|
@@ -9938,7 +9938,7 @@
|
|
|
9938
9938
|
"kind": "interface",
|
|
9939
9939
|
"locationInModule": {
|
|
9940
9940
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9941
|
-
"line":
|
|
9941
|
+
"line": 79
|
|
9942
9942
|
},
|
|
9943
9943
|
"name": "IcebergColumn",
|
|
9944
9944
|
"properties": [
|
|
@@ -9952,7 +9952,7 @@
|
|
|
9952
9952
|
"immutable": true,
|
|
9953
9953
|
"locationInModule": {
|
|
9954
9954
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9955
|
-
"line":
|
|
9955
|
+
"line": 100
|
|
9956
9956
|
},
|
|
9957
9957
|
"name": "id",
|
|
9958
9958
|
"type": {
|
|
@@ -9968,7 +9968,7 @@
|
|
|
9968
9968
|
"immutable": true,
|
|
9969
9969
|
"locationInModule": {
|
|
9970
9970
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9971
|
-
"line":
|
|
9971
|
+
"line": 81
|
|
9972
9972
|
},
|
|
9973
9973
|
"name": "name",
|
|
9974
9974
|
"type": {
|
|
@@ -9984,7 +9984,7 @@
|
|
|
9984
9984
|
"immutable": true,
|
|
9985
9985
|
"locationInModule": {
|
|
9986
9986
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9987
|
-
"line":
|
|
9987
|
+
"line": 84
|
|
9988
9988
|
},
|
|
9989
9989
|
"name": "type",
|
|
9990
9990
|
"type": {
|
|
@@ -10001,7 +10001,7 @@
|
|
|
10001
10001
|
"immutable": true,
|
|
10002
10002
|
"locationInModule": {
|
|
10003
10003
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10004
|
-
"line":
|
|
10004
|
+
"line": 93
|
|
10005
10005
|
},
|
|
10006
10006
|
"name": "doc",
|
|
10007
10007
|
"optional": true,
|
|
@@ -10019,7 +10019,7 @@
|
|
|
10019
10019
|
"immutable": true,
|
|
10020
10020
|
"locationInModule": {
|
|
10021
10021
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10022
|
-
"line":
|
|
10022
|
+
"line": 90
|
|
10023
10023
|
},
|
|
10024
10024
|
"name": "required",
|
|
10025
10025
|
"optional": true,
|
|
@@ -10333,7 +10333,7 @@
|
|
|
10333
10333
|
"kind": "enum",
|
|
10334
10334
|
"locationInModule": {
|
|
10335
10335
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10336
|
-
"line":
|
|
10336
|
+
"line": 66
|
|
10337
10337
|
},
|
|
10338
10338
|
"members": [
|
|
10339
10339
|
{
|
|
@@ -10365,7 +10365,7 @@
|
|
|
10365
10365
|
"kind": "interface",
|
|
10366
10366
|
"locationInModule": {
|
|
10367
10367
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10368
|
-
"line":
|
|
10368
|
+
"line": 104
|
|
10369
10369
|
},
|
|
10370
10370
|
"name": "IcebergPartitionField",
|
|
10371
10371
|
"properties": [
|
|
@@ -10379,7 +10379,7 @@
|
|
|
10379
10379
|
"immutable": true,
|
|
10380
10380
|
"locationInModule": {
|
|
10381
10381
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10382
|
-
"line":
|
|
10382
|
+
"line": 135
|
|
10383
10383
|
},
|
|
10384
10384
|
"name": "fieldId",
|
|
10385
10385
|
"type": {
|
|
@@ -10395,7 +10395,7 @@
|
|
|
10395
10395
|
"immutable": true,
|
|
10396
10396
|
"locationInModule": {
|
|
10397
10397
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10398
|
-
"line":
|
|
10398
|
+
"line": 106
|
|
10399
10399
|
},
|
|
10400
10400
|
"name": "sourceColumn",
|
|
10401
10401
|
"type": {
|
|
@@ -10411,7 +10411,7 @@
|
|
|
10411
10411
|
"immutable": true,
|
|
10412
10412
|
"locationInModule": {
|
|
10413
10413
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10414
|
-
"line":
|
|
10414
|
+
"line": 109
|
|
10415
10415
|
},
|
|
10416
10416
|
"name": "transform",
|
|
10417
10417
|
"type": {
|
|
@@ -10428,7 +10428,7 @@
|
|
|
10428
10428
|
"immutable": true,
|
|
10429
10429
|
"locationInModule": {
|
|
10430
10430
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10431
|
-
"line":
|
|
10431
|
+
"line": 116
|
|
10432
10432
|
},
|
|
10433
10433
|
"name": "name",
|
|
10434
10434
|
"optional": true,
|
|
@@ -11105,7 +11105,7 @@
|
|
|
11105
11105
|
"kind": "enum",
|
|
11106
11106
|
"locationInModule": {
|
|
11107
11107
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11108
|
-
"line":
|
|
11108
|
+
"line": 57
|
|
11109
11109
|
},
|
|
11110
11110
|
"members": [
|
|
11111
11111
|
{
|
|
@@ -11137,7 +11137,7 @@
|
|
|
11137
11137
|
"kind": "interface",
|
|
11138
11138
|
"locationInModule": {
|
|
11139
11139
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11140
|
-
"line":
|
|
11140
|
+
"line": 139
|
|
11141
11141
|
},
|
|
11142
11142
|
"name": "IcebergSortField",
|
|
11143
11143
|
"properties": [
|
|
@@ -11150,7 +11150,7 @@
|
|
|
11150
11150
|
"immutable": true,
|
|
11151
11151
|
"locationInModule": {
|
|
11152
11152
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11153
|
-
"line":
|
|
11153
|
+
"line": 141
|
|
11154
11154
|
},
|
|
11155
11155
|
"name": "sourceColumn",
|
|
11156
11156
|
"type": {
|
|
@@ -11167,7 +11167,7 @@
|
|
|
11167
11167
|
"immutable": true,
|
|
11168
11168
|
"locationInModule": {
|
|
11169
11169
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11170
|
-
"line":
|
|
11170
|
+
"line": 147
|
|
11171
11171
|
},
|
|
11172
11172
|
"name": "direction",
|
|
11173
11173
|
"optional": true,
|
|
@@ -11185,7 +11185,7 @@
|
|
|
11185
11185
|
"immutable": true,
|
|
11186
11186
|
"locationInModule": {
|
|
11187
11187
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11188
|
-
"line":
|
|
11188
|
+
"line": 150
|
|
11189
11189
|
},
|
|
11190
11190
|
"name": "nullOrder",
|
|
11191
11191
|
"optional": true,
|
|
@@ -11203,7 +11203,7 @@
|
|
|
11203
11203
|
"immutable": true,
|
|
11204
11204
|
"locationInModule": {
|
|
11205
11205
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11206
|
-
"line":
|
|
11206
|
+
"line": 144
|
|
11207
11207
|
},
|
|
11208
11208
|
"name": "transform",
|
|
11209
11209
|
"optional": true,
|
|
@@ -11330,7 +11330,7 @@
|
|
|
11330
11330
|
},
|
|
11331
11331
|
"locationInModule": {
|
|
11332
11332
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11333
|
-
"line":
|
|
11333
|
+
"line": 282
|
|
11334
11334
|
},
|
|
11335
11335
|
"parameters": [
|
|
11336
11336
|
{
|
|
@@ -11359,9 +11359,47 @@
|
|
|
11359
11359
|
"kind": "class",
|
|
11360
11360
|
"locationInModule": {
|
|
11361
11361
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11362
|
-
"line":
|
|
11362
|
+
"line": 256
|
|
11363
11363
|
},
|
|
11364
11364
|
"methods": [
|
|
11365
|
+
{
|
|
11366
|
+
"docs": {
|
|
11367
|
+
"stability": "experimental",
|
|
11368
|
+
"summary": "Field ids Glue assigns when it creates a table with these columns, keyed by field path (`addr.city`, `tags.element`, `attrs.key`, `attrs.value`). Declare a new table with these ids; the ids in `columns` are ignored. A name containing `.` or a backquote appears in backquotes, inner backquotes doubled. From Python, a top-level column named `__proto__` makes the call raise a jsii serialization error."
|
|
11369
|
+
},
|
|
11370
|
+
"locationInModule": {
|
|
11371
|
+
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11372
|
+
"line": 357
|
|
11373
|
+
},
|
|
11374
|
+
"name": "creationOrderFieldIds",
|
|
11375
|
+
"parameters": [
|
|
11376
|
+
{
|
|
11377
|
+
"docs": {
|
|
11378
|
+
"summary": "Columns of the table, in declaration order."
|
|
11379
|
+
},
|
|
11380
|
+
"name": "columns",
|
|
11381
|
+
"type": {
|
|
11382
|
+
"collection": {
|
|
11383
|
+
"elementtype": {
|
|
11384
|
+
"fqn": "cdk-glue-iceberg-table.IcebergColumn"
|
|
11385
|
+
},
|
|
11386
|
+
"kind": "array"
|
|
11387
|
+
}
|
|
11388
|
+
}
|
|
11389
|
+
}
|
|
11390
|
+
],
|
|
11391
|
+
"returns": {
|
|
11392
|
+
"type": {
|
|
11393
|
+
"collection": {
|
|
11394
|
+
"elementtype": {
|
|
11395
|
+
"primitive": "number"
|
|
11396
|
+
},
|
|
11397
|
+
"kind": "map"
|
|
11398
|
+
}
|
|
11399
|
+
}
|
|
11400
|
+
},
|
|
11401
|
+
"static": true
|
|
11402
|
+
},
|
|
11365
11403
|
{
|
|
11366
11404
|
"docs": {
|
|
11367
11405
|
"remarks": "Imported tables can be referenced from grants but cannot be\nmutated; their schema and properties are unknown to CDK.",
|
|
@@ -11370,7 +11408,7 @@
|
|
|
11370
11408
|
},
|
|
11371
11409
|
"locationInModule": {
|
|
11372
11410
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11373
|
-
"line":
|
|
11411
|
+
"line": 367
|
|
11374
11412
|
},
|
|
11375
11413
|
"name": "fromIcebergTableAttributes",
|
|
11376
11414
|
"parameters": [
|
|
@@ -11407,7 +11445,7 @@
|
|
|
11407
11445
|
},
|
|
11408
11446
|
"locationInModule": {
|
|
11409
11447
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11410
|
-
"line":
|
|
11448
|
+
"line": 446
|
|
11411
11449
|
},
|
|
11412
11450
|
"name": "grantRead",
|
|
11413
11451
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -11432,7 +11470,7 @@
|
|
|
11432
11470
|
},
|
|
11433
11471
|
"locationInModule": {
|
|
11434
11472
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11435
|
-
"line":
|
|
11473
|
+
"line": 474
|
|
11436
11474
|
},
|
|
11437
11475
|
"name": "grantReadWrite",
|
|
11438
11476
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -11457,7 +11495,7 @@
|
|
|
11457
11495
|
},
|
|
11458
11496
|
"locationInModule": {
|
|
11459
11497
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11460
|
-
"line":
|
|
11498
|
+
"line": 460
|
|
11461
11499
|
},
|
|
11462
11500
|
"name": "grantWrite",
|
|
11463
11501
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -11486,7 +11524,7 @@
|
|
|
11486
11524
|
"immutable": true,
|
|
11487
11525
|
"locationInModule": {
|
|
11488
11526
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11489
|
-
"line":
|
|
11527
|
+
"line": 264
|
|
11490
11528
|
},
|
|
11491
11529
|
"name": "database",
|
|
11492
11530
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -11502,7 +11540,7 @@
|
|
|
11502
11540
|
"immutable": true,
|
|
11503
11541
|
"locationInModule": {
|
|
11504
11542
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11505
|
-
"line":
|
|
11543
|
+
"line": 270
|
|
11506
11544
|
},
|
|
11507
11545
|
"name": "dataFormat",
|
|
11508
11546
|
"type": {
|
|
@@ -11517,7 +11555,7 @@
|
|
|
11517
11555
|
"immutable": true,
|
|
11518
11556
|
"locationInModule": {
|
|
11519
11557
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11520
|
-
"line":
|
|
11558
|
+
"line": 273
|
|
11521
11559
|
},
|
|
11522
11560
|
"name": "formatVersion",
|
|
11523
11561
|
"type": {
|
|
@@ -11532,7 +11570,7 @@
|
|
|
11532
11570
|
"immutable": true,
|
|
11533
11571
|
"locationInModule": {
|
|
11534
11572
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11535
|
-
"line":
|
|
11573
|
+
"line": 267
|
|
11536
11574
|
},
|
|
11537
11575
|
"name": "location",
|
|
11538
11576
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -11548,7 +11586,7 @@
|
|
|
11548
11586
|
"immutable": true,
|
|
11549
11587
|
"locationInModule": {
|
|
11550
11588
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11551
|
-
"line":
|
|
11589
|
+
"line": 276
|
|
11552
11590
|
},
|
|
11553
11591
|
"name": "resource",
|
|
11554
11592
|
"type": {
|
|
@@ -11563,7 +11601,7 @@
|
|
|
11563
11601
|
"immutable": true,
|
|
11564
11602
|
"locationInModule": {
|
|
11565
11603
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11566
|
-
"line":
|
|
11604
|
+
"line": 258
|
|
11567
11605
|
},
|
|
11568
11606
|
"name": "tableArn",
|
|
11569
11607
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -11579,7 +11617,7 @@
|
|
|
11579
11617
|
"immutable": true,
|
|
11580
11618
|
"locationInModule": {
|
|
11581
11619
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11582
|
-
"line":
|
|
11620
|
+
"line": 261
|
|
11583
11621
|
},
|
|
11584
11622
|
"name": "tableName",
|
|
11585
11623
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -11601,7 +11639,7 @@
|
|
|
11601
11639
|
"kind": "interface",
|
|
11602
11640
|
"locationInModule": {
|
|
11603
11641
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11604
|
-
"line":
|
|
11642
|
+
"line": 498
|
|
11605
11643
|
},
|
|
11606
11644
|
"name": "IcebergTableAttributes",
|
|
11607
11645
|
"properties": [
|
|
@@ -11614,7 +11652,7 @@
|
|
|
11614
11652
|
"immutable": true,
|
|
11615
11653
|
"locationInModule": {
|
|
11616
11654
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11617
|
-
"line":
|
|
11655
|
+
"line": 500
|
|
11618
11656
|
},
|
|
11619
11657
|
"name": "database",
|
|
11620
11658
|
"type": {
|
|
@@ -11630,7 +11668,7 @@
|
|
|
11630
11668
|
"immutable": true,
|
|
11631
11669
|
"locationInModule": {
|
|
11632
11670
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11633
|
-
"line":
|
|
11671
|
+
"line": 506
|
|
11634
11672
|
},
|
|
11635
11673
|
"name": "location",
|
|
11636
11674
|
"type": {
|
|
@@ -11646,7 +11684,7 @@
|
|
|
11646
11684
|
"immutable": true,
|
|
11647
11685
|
"locationInModule": {
|
|
11648
11686
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11649
|
-
"line":
|
|
11687
|
+
"line": 503
|
|
11650
11688
|
},
|
|
11651
11689
|
"name": "tableName",
|
|
11652
11690
|
"type": {
|
|
@@ -11667,7 +11705,7 @@
|
|
|
11667
11705
|
"kind": "interface",
|
|
11668
11706
|
"locationInModule": {
|
|
11669
11707
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11670
|
-
"line":
|
|
11708
|
+
"line": 154
|
|
11671
11709
|
},
|
|
11672
11710
|
"name": "IcebergTableProps",
|
|
11673
11711
|
"properties": [
|
|
@@ -11681,7 +11719,7 @@
|
|
|
11681
11719
|
"immutable": true,
|
|
11682
11720
|
"locationInModule": {
|
|
11683
11721
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11684
|
-
"line":
|
|
11722
|
+
"line": 162
|
|
11685
11723
|
},
|
|
11686
11724
|
"name": "columns",
|
|
11687
11725
|
"type": {
|
|
@@ -11702,7 +11740,7 @@
|
|
|
11702
11740
|
"immutable": true,
|
|
11703
11741
|
"locationInModule": {
|
|
11704
11742
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11705
|
-
"line":
|
|
11743
|
+
"line": 156
|
|
11706
11744
|
},
|
|
11707
11745
|
"name": "database",
|
|
11708
11746
|
"type": {
|
|
@@ -11719,7 +11757,7 @@
|
|
|
11719
11757
|
"immutable": true,
|
|
11720
11758
|
"locationInModule": {
|
|
11721
11759
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11722
|
-
"line":
|
|
11760
|
+
"line": 188
|
|
11723
11761
|
},
|
|
11724
11762
|
"name": "formatVersion",
|
|
11725
11763
|
"type": {
|
|
@@ -11736,7 +11774,7 @@
|
|
|
11736
11774
|
"immutable": true,
|
|
11737
11775
|
"locationInModule": {
|
|
11738
11776
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11739
|
-
"line":
|
|
11777
|
+
"line": 165
|
|
11740
11778
|
},
|
|
11741
11779
|
"name": "location",
|
|
11742
11780
|
"type": {
|
|
@@ -11752,7 +11790,7 @@
|
|
|
11752
11790
|
"immutable": true,
|
|
11753
11791
|
"locationInModule": {
|
|
11754
11792
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11755
|
-
"line":
|
|
11793
|
+
"line": 159
|
|
11756
11794
|
},
|
|
11757
11795
|
"name": "tableName",
|
|
11758
11796
|
"type": {
|
|
@@ -11769,7 +11807,7 @@
|
|
|
11769
11807
|
"immutable": true,
|
|
11770
11808
|
"locationInModule": {
|
|
11771
11809
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11772
|
-
"line":
|
|
11810
|
+
"line": 208
|
|
11773
11811
|
},
|
|
11774
11812
|
"name": "comment",
|
|
11775
11813
|
"optional": true,
|
|
@@ -11787,7 +11825,7 @@
|
|
|
11787
11825
|
"immutable": true,
|
|
11788
11826
|
"locationInModule": {
|
|
11789
11827
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11790
|
-
"line":
|
|
11828
|
+
"line": 185
|
|
11791
11829
|
},
|
|
11792
11830
|
"name": "dataFormat",
|
|
11793
11831
|
"optional": true,
|
|
@@ -11805,7 +11843,7 @@
|
|
|
11805
11843
|
"immutable": true,
|
|
11806
11844
|
"locationInModule": {
|
|
11807
11845
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11808
|
-
"line":
|
|
11846
|
+
"line": 182
|
|
11809
11847
|
},
|
|
11810
11848
|
"name": "identifierFieldNames",
|
|
11811
11849
|
"optional": true,
|
|
@@ -11828,7 +11866,7 @@
|
|
|
11828
11866
|
"immutable": true,
|
|
11829
11867
|
"locationInModule": {
|
|
11830
11868
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11831
|
-
"line":
|
|
11869
|
+
"line": 168
|
|
11832
11870
|
},
|
|
11833
11871
|
"name": "partitionSpec",
|
|
11834
11872
|
"optional": true,
|
|
@@ -11851,7 +11889,7 @@
|
|
|
11851
11889
|
"immutable": true,
|
|
11852
11890
|
"locationInModule": {
|
|
11853
11891
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11854
|
-
"line":
|
|
11892
|
+
"line": 215
|
|
11855
11893
|
},
|
|
11856
11894
|
"name": "removalPolicy",
|
|
11857
11895
|
"optional": true,
|
|
@@ -11869,7 +11907,7 @@
|
|
|
11869
11907
|
"immutable": true,
|
|
11870
11908
|
"locationInModule": {
|
|
11871
11909
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11872
|
-
"line":
|
|
11910
|
+
"line": 171
|
|
11873
11911
|
},
|
|
11874
11912
|
"name": "sortOrder",
|
|
11875
11913
|
"optional": true,
|
|
@@ -11892,7 +11930,7 @@
|
|
|
11892
11930
|
"immutable": true,
|
|
11893
11931
|
"locationInModule": {
|
|
11894
11932
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
11895
|
-
"line":
|
|
11933
|
+
"line": 197
|
|
11896
11934
|
},
|
|
11897
11935
|
"name": "tableProperties",
|
|
11898
11936
|
"optional": true,
|
|
@@ -13194,6 +13232,6 @@
|
|
|
13194
13232
|
"symbolId": "lib/iceberg/iceberg-type:IcebergTypeOptions"
|
|
13195
13233
|
}
|
|
13196
13234
|
},
|
|
13197
|
-
"version": "0.
|
|
13198
|
-
"fingerprint": "
|
|
13235
|
+
"version": "0.14.0",
|
|
13236
|
+
"fingerprint": "CgLRWE9zEd8Gw1LIIoZ6FO3yxtevWLhxoIbpa7wPSwc="
|
|
13199
13237
|
}
|