cdk-glue-iceberg-table 0.5.0 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.jsii +66 -49
- package/README.md +50 -12
- package/dist/lib/iceberg/iceberg-partition-transform.js +1 -1
- package/dist/lib/iceberg/iceberg-table-properties.js +1 -1
- package/dist/lib/iceberg/iceberg-table-render.js +17 -8
- package/dist/lib/iceberg/iceberg-table.d.ts +18 -0
- package/dist/lib/iceberg/iceberg-table.js +2 -2
- package/dist/lib/iceberg/iceberg-type.d.ts +1 -1
- package/dist/lib/iceberg/iceberg-type.js +15 -3
- package/package.json +1 -1
package/.jsii
CHANGED
|
@@ -9070,7 +9070,7 @@
|
|
|
9070
9070
|
},
|
|
9071
9071
|
"name": "cdk-glue-iceberg-table",
|
|
9072
9072
|
"readme": {
|
|
9073
|
-
"markdown": "# cdk-glue-iceberg-table\n\n[](https://github.com/ksco92/arceus/actions/workflows/ci.yml)\n[](https://codecov.io/gh/ksco92/arceus)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://pypi.org/project/cdk-glue-iceberg-table/)\n[](https://constructs.dev/packages/cdk-glue-iceberg-table)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://github.com/ksco92/arceus/commits/main)\n[](LICENSE)\n\n**`cdk-glue-iceberg-table` is the AWS CDK L2 construct for creating\nand evolving Apache Iceberg tables in the AWS Glue Data Catalog.** It\nemits the exact `AWS::Glue::Table` / `OpenTableFormatInput` shape that\nsurvives a CloudFormation `Update`, so a single `cdk deploy` creates a\ntable, evolves its schema and partitions, and destroys it, the same\nlifecycle CDK gives any other resource. No custom resource, no Lambda,\nno \"`cdk deploy` and then run this SQL by hand\" two-step. Table\nchanges (new columns, renames, drops, new partition fields) land as\na reviewed diff in a pull request and apply through `cdk deploy`.\n\nStatus (June 2026): published on npm (TypeScript/JavaScript) and PyPI\n(Python), pre-1.0, with a public surface guarded by an end-to-end\nconsumer test on every PR plus three real-AWS integration suites. The\nmulti-language packages are generated from one TypeScript source with\n[jsii](https://aws.github.io/jsii/) and indexed on\n[Construct Hub](https://constructs.dev/packages/cdk-glue-iceberg-table).\nDeveloped in the\n[`ksco92/arceus`](https://github.com/ksco92/arceus) monorepo, which\nalso holds a CDK demo app that dogfoods the construct against a real\nAWS account.\n\n### Why this exists\n\nAWS documents the CloudFormation shape for Iceberg tables ([AWS Big\nData blog, December 2025](https://aws.amazon.com/blogs/big-data/create-and-update-apache-iceberg-tables-with-partitions-in-the-aws-glue-data-catalog-using-the-aws-sdk-and-aws-cloudformation)),\nbut raw `AWS::Glue::Table` is a minefield: the CREATE succeeds and the\n*first* `Update` silently strips `table_type=ICEBERG`, after which\nevery Athena query fails with `HIVE_UNSUPPORTED_FORMAT`. The\nmotivating CDK issue is [aws/aws-cdk#29660](https://github.com/aws/aws-cdk/issues/29660);\n[`manmartgarc`'s comment](https://github.com/aws/aws-cdk/issues/29660#issuecomment-4411359248)\ndocuments the only working shape and the silent-corruption traps you\nhit by getting it slightly wrong. This construct implements that\nshape, refuses to emit the unsafe alternatives, and pins Iceberg field\nIDs so schema evolution never orphans existing data. It is the basis\nof the upstream proposal to land an `IcebergTable` in\n`@aws-cdk/aws-glue-alpha` ([aws/aws-cdk#37988](https://github.com/aws/aws-cdk/pull/37988));\nthis package tracks that proposal and stays current with it.\n\n### How it compares\n\n| Approach | Declarative IaC (in the PR) | In-place schema + partition evolution | Prevents the silent-corruption footguns | Typed API + synth-time validation |\n| --- | --- | --- | --- | --- |\n| Raw `CfnTable` (L1) | ✅ | ⚠️ only if you hand-write the exact `OpenTableFormatInput` shape | ❌ you own both footguns | ❌ |\n| CDK custom resource (Lambda + Glue SDK) | ⚠️ via a custom resource you maintain | ⚠️ you write the diff logic | ⚠️ your code's responsibility | ❌ |\n| Spark / Athena SQL DDL at job runtime | ❌ imperative, outside the PR | ✅ but outside CloudFormation | n/a | ❌ |\n| **`cdk-glue-iceberg-table`** | ✅ | ✅ via `cdk deploy` | ✅ both prevented by construction | ✅ |\n\n## Install\n\nTypeScript / JavaScript (npm):\n\n```bash\nnpm install cdk-glue-iceberg-table\n```\n\nPython (PyPI):\n\n```bash\npip install cdk-glue-iceberg-table\n```\n\nPeer dependencies (your CDK app must already have these). For\nTypeScript / JavaScript:\n\n```bash\nnpm install aws-cdk-lib constructs @aws-cdk/aws-glue-alpha\n```\n\nFor Python:\n\n```bash\npip install aws-cdk-lib constructs aws-cdk.aws-glue-alpha\n```\n\n## Use\n\n### TypeScript\n\n```ts\nimport { Bucket } from 'aws-cdk-lib/aws-s3';\nimport { Database } from '@aws-cdk/aws-glue-alpha';\nimport {\n IcebergTable,\n IcebergType,\n IcebergPartitionTransform,\n} from 'cdk-glue-iceberg-table';\n\nconst bucket = new Bucket(this, 'Warehouse');\nconst db = new Database(this, 'Db', { databaseName: 'analytics' });\n\nnew IcebergTable(this, 'OrdersTable', {\n database: db,\n tableName: 'orders',\n location: `s3://${bucket.bucketName}/analytics/orders/`,\n columns: [\n { name: 'order_id', type: IcebergType.LONG, required: true, id: 1 },\n { name: 'customer_id', type: IcebergType.LONG, required: true, id: 2 },\n { name: 'placed_at', type: IcebergType.TIMESTAMPTZ, required: true, id: 3 },\n ],\n partitionSpec: [\n { sourceColumn: 'placed_at', transform: IcebergPartitionTransform.DAY },\n { sourceColumn: 'customer_id', transform: IcebergPartitionTransform.bucket(16) },\n ],\n identifierFieldNames: ['order_id'],\n});\n```\n\n### Python\n\nThe same table in Python — the API mirrors TypeScript with\n`snake_case` props and `PascalCase` types:\n\n```python\nfrom aws_cdk.aws_s3 import Bucket\nfrom aws_cdk.aws_glue_alpha import Database\nfrom cdk_glue_iceberg_table import (\n IcebergTable,\n IcebergType,\n IcebergPartitionTransform,\n)\n\nbucket = Bucket(self, \"Warehouse\")\ndb = Database(self, \"Db\", database_name=\"analytics\")\n\nIcebergTable(self, \"OrdersTable\",\n database=db,\n table_name=\"orders\",\n location=f\"s3://{bucket.bucket_name}/analytics/orders/\",\n columns=[\n {\"name\": \"order_id\", \"type\": IcebergType.LONG, \"required\": True, \"id\": 1},\n {\"name\": \"customer_id\", \"type\": IcebergType.LONG, \"required\": True, \"id\": 2},\n {\"name\": \"placed_at\", \"type\": IcebergType.TIMESTAMPTZ, \"required\": True, \"id\": 3},\n ],\n partition_spec=[\n {\"source_column\": \"placed_at\", \"transform\": IcebergPartitionTransform.DAY},\n {\"source_column\": \"customer_id\", \"transform\": IcebergPartitionTransform.bucket(16)},\n ],\n identifier_field_names=[\"order_id\"],\n)\n```\n\nConsumer-facing reference sections:\n\n- [How it compares](#how-it-compares) — `cdk-glue-iceberg-table` vs raw `CfnTable`, custom resources, and runtime SQL DDL.\n- [Using `IcebergTable`](#using-icebergtable) — full API reference with examples.\n- [Two footguns the construct prevents](#two-footguns-the-construct-prevents) — the silent-corruption traps that motivated this construct.\n- [Known limitations](#known-limitations) — what the construct does and doesn't enforce.\n- [FAQ](#faq) — common \"how do I … in CDK / CloudFormation\" questions.\n\n## Exported surface\n\nThe package entry point re-exports everything you import from\n`cdk-glue-iceberg-table`. In TypeScript / JavaScript the compiled\nsurface lives under `dist/lib/iceberg/` after `npm install`; in Python\nthe same surface is imported from the `cdk_glue_iceberg_table` module\nafter `pip install`:\n\n- **`IcebergTable`** — the L2 construct itself, with `grantRead` / `grantWrite` / `grantReadWrite` and the `fromIcebergTableAttributes(...)` import factory.\n- **`IcebergType`** — primitive types plus `list` / `map` / `struct` / `decimal` / `fixed` factories. Renders to the JSON shape Glue's `IcebergStructField.type` expects.\n- **`IcebergPartitionTransform`** — identity / `bucket(N)` / `truncate(W)` / year / month / day / hour / void. Each transform validates against the source column type at synth time.\n- **`IcebergDataFormat`** (parquet/orc/avro, default parquet), **`IcebergFormatVersion`** (v1/v2, required — set explicitly per table), **`IcebergSortDirection`**, **`IcebergNullOrder`**, plus a `tableProperties` validator that catches misconfigured properties before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).\n\n## Using `IcebergTable`\n\nA minimal table:\n\n```typescript\nimport {\n Database,\n} from '@aws-cdk/aws-glue-alpha';\nimport {\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nconst db = new Database(this, 'Db', {\n databaseName: 'analytics',\n});\n\nnew IcebergTable(this, 'Users', {\n database: db,\n tableName: 'users',\n columns: [\n {\n name: 'user_id',\n type: IcebergType.LONG,\n required: true,\n id: 1,\n },\n {\n name: 'email',\n type: IcebergType.STRING,\n required: true,\n id: 2,\n },\n {\n name: 'signed_up_at',\n type: IcebergType.TIMESTAMPTZ,\n required: true,\n id: 3,\n },\n ],\n location: `s3://${bucket.bucketName}/analytics/users/`,\n});\n```\n\nEvery top-level column must declare an `id`: a stable, unique,\npositive integer you pick once and never reuse or reassign. Because\nIceberg data files reference fields by id rather than by name, a fixed\nid is what lets you add, rename, and drop columns across deploys\nwithout corrupting old data — see\n[How do I evolve an Iceberg table schema](#how-do-i-evolve-an-iceberg-table-schema-add-rename-or-drop-a-column-in-cloudformation).\nOnce a column is dropped, leave its id retired; never hand it to a new\ncolumn. (Nested struct/list/map field ids are assigned automatically\nabove the highest top-level id — only top-level column ids are\ncaller-provided.)\n\nA table that exercises most of the surface (partitions, sort order,\nnested types, identifier fields, table properties, removal policy):\n\n```typescript\nimport {\n RemovalPolicy,\n} from 'aws-cdk-lib';\nimport {\n Database,\n} from '@aws-cdk/aws-glue-alpha';\nimport {\n IcebergDataFormat,\n IcebergFormatVersion,\n IcebergNullOrder,\n IcebergPartitionTransform,\n IcebergSortDirection,\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nnew IcebergTable(this, 'OrdersTable', {\n database: db,\n tableName: 'orders',\n comment: 'Demo Iceberg orders table — exercises partitions, sort order, and merge-on-read.',\n columns: [\n {\n name: 'order_id',\n type: IcebergType.LONG,\n required: true,\n id: 1,\n },\n {\n name: 'customer_id',\n type: IcebergType.LONG,\n required: true,\n id: 2,\n },\n {\n name: 'order_amount',\n type: IcebergType.decimal(12, 2),\n required: true,\n id: 3,\n },\n {\n name: 'currency',\n type: IcebergType.STRING,\n required: true,\n id: 4,\n },\n {\n name: 'placed_at',\n type: IcebergType.TIMESTAMPTZ,\n required: true,\n id: 5,\n },\n {\n name: 'tags',\n type: IcebergType.list(IcebergType.STRING),\n id: 6,\n },\n {\n name: 'shipping_address',\n type: IcebergType.struct([\n {\n name: 'line1',\n type: IcebergType.STRING,\n required: true,\n },\n {\n name: 'city',\n type: IcebergType.STRING,\n required: true,\n },\n {\n name: 'country',\n type: IcebergType.STRING,\n required: true,\n },\n {\n name: 'postal_code',\n type: IcebergType.STRING,\n },\n ]),\n id: 7,\n },\n {\n name: 'metadata',\n type: IcebergType.map(IcebergType.STRING, IcebergType.STRING, false),\n id: 8,\n },\n ],\n location: `s3://${bucket.bucketName}/analytics/orders/`,\n partitionSpec: [\n {\n sourceColumn: 'placed_at',\n transform: IcebergPartitionTransform.DAY,\n },\n {\n sourceColumn: 'customer_id',\n transform: IcebergPartitionTransform.bucket(16),\n },\n ],\n sortOrder: [\n {\n sourceColumn: 'placed_at',\n direction: IcebergSortDirection.ASC,\n nullOrder: IcebergNullOrder.NULLS_LAST,\n },\n {\n sourceColumn: 'order_id',\n direction: IcebergSortDirection.ASC,\n },\n ],\n identifierFieldNames: [\n 'order_id',\n ],\n dataFormat: IcebergDataFormat.PARQUET,\n formatVersion: IcebergFormatVersion.V2,\n tableProperties: {\n 'write.parquet.compression-codec': 'zstd',\n 'write.delete.mode': 'merge-on-read',\n 'write.update.mode': 'merge-on-read',\n 'write.merge.mode': 'merge-on-read',\n 'write.target-file-size-bytes': '134217728',\n 'history.expire.min-snapshots-to-keep': '5',\n 'gc.enabled': 'true',\n },\n removalPolicy: RemovalPolicy.DESTROY,\n});\n```\n\nThe resulting Iceberg `metadata.json` for this table contains every\nfeature you set:\n\n```json\n{\n \"format-version\": 2,\n \"table-uuid\": \"39a948f9-...\",\n \"current-schema-id\": 0,\n \"schemas\": [\n {\n \"schema-id\": 0,\n \"identifier-field-ids\": [1],\n \"fields\": [\n { \"id\": 1, \"name\": \"order_id\", \"required\": true, \"type\": \"long\" },\n { \"id\": 2, \"name\": \"customer_id\", \"required\": true, \"type\": \"long\" },\n { \"id\": 3, \"name\": \"order_amount\", \"required\": true, \"type\": \"decimal(12, 2)\" },\n { \"id\": 4, \"name\": \"currency\", \"required\": true, \"type\": \"string\" },\n { \"id\": 5, \"name\": \"placed_at\", \"required\": true, \"type\": \"timestamptz\" },\n { \"id\": 6, \"name\": \"tags\", \"required\": false,\n \"type\": { \"type\": \"list\", \"element-id\": 9, \"element\": \"string\", \"element-required\": true } },\n { \"id\": 7, \"name\": \"shipping_address\", \"required\": false,\n \"type\": { \"type\": \"struct\", \"fields\": [\n { \"id\": 10, \"name\": \"line1\", \"required\": true, \"type\": \"string\" },\n { \"id\": 11, \"name\": \"city\", \"required\": true, \"type\": \"string\" },\n { \"id\": 12, \"name\": \"country\", \"required\": true, \"type\": \"string\" },\n { \"id\": 13, \"name\": \"postal_code\", \"required\": false, \"type\": \"string\" }\n ] } },\n { \"id\": 8, \"name\": \"metadata\", \"required\": false,\n \"type\": { \"type\": \"map\", \"key-id\": 14, \"key\": \"string\", \"value-id\": 15,\n \"value-required\": false, \"value\": \"string\" } }\n ]\n }\n ],\n \"partition-specs\": [\n { \"spec-id\": 0, \"fields\": [\n { \"name\": \"placed_at_day\", \"transform\": \"day\", \"source-id\": 5, \"field-id\": 1000 },\n { \"name\": \"customer_id_bucket\", \"transform\": \"bucket[16]\", \"source-id\": 2, \"field-id\": 1001 }\n ]}\n ],\n \"sort-orders\": [\n { \"order-id\": 1, \"fields\": [\n { \"transform\": \"identity\", \"source-id\": 5, \"direction\": \"asc\", \"null-order\": \"nulls-last\" },\n { \"transform\": \"identity\", \"source-id\": 1, \"direction\": \"asc\", \"null-order\": \"nulls-last\" }\n ]}\n ],\n \"properties\": {\n \"format-version\": \"2\",\n \"write.format.default\": \"parquet\",\n \"write.parquet.compression-codec\": \"zstd\",\n \"write.merge.mode\": \"merge-on-read\",\n \"write.update.mode\": \"merge-on-read\",\n \"write.delete.mode\": \"merge-on-read\",\n \"write.target-file-size-bytes\": \"134217728\",\n \"history.expire.min-snapshots-to-keep\": \"5\",\n \"gc.enabled\": \"true\",\n \"comment\": \"Demo Iceberg orders table — exercises partitions, sort order, and merge-on-read.\"\n }\n}\n```\n\n### Granting access\n\n```typescript\ntable.grantRead(role); // Glue read + S3 read on the table's prefix\ntable.grantWrite(role); // Glue write + S3 write\ntable.grantReadWrite(role);\n```\n\nThe `grant*` helpers issue **IAM grants only**. Under Lake Formation\nyou still add the matching `SELECT` / `INSERT` / `DELETE` LF grants on\ntop of the construct's IAM grants for Athena queries to succeed.\n\n### Importing an existing table\n\n```typescript\nconst existing = IcebergTable.fromIcebergTableAttributes(this, 'Orders', {\n database: db,\n tableName: 'orders',\n location: 's3://my-bucket/analytics/orders/',\n});\nexisting.grantRead(role);\n```\n\n### Evolving schema and partitions\n\nChange the `columns` array (or `partitionSpec`) and run `cdk deploy`\nagain. The construct passes the new schema to Glue's `UpdateTable`,\nwhich writes a new `metadata.json` with a new `schema-id`; existing\ndata files stay readable because each column's `id` is pinned and\nnever reused. Adds, renames (same `id`, new `name`), and drops all\nflow through `cdk deploy` alone — no out-of-band SQL DDL. The same\napplies to `partitionSpec`.\n\n### Inserting and querying\n\n```sql\n-- INSERT into the orders table\nINSERT INTO sample_database.orders VALUES\n (1001, 5001, DECIMAL '149.99', 'USD',\n TIMESTAMP '2026-05-20 09:15:00 UTC',\n ARRAY['holiday-promo','first-order'],\n CAST(ROW('1 Infinite Loop','Cupertino','US','95014')\n AS ROW(line1 VARCHAR,city VARCHAR,country VARCHAR,postal_code VARCHAR)),\n MAP(ARRAY['channel','utm'], ARRAY['web','google'])),\n -- ... more rows\n;\n\n-- merge-on-read DELETE (only legal because we chose v2 + merge-on-read mode)\nDELETE FROM sample_database.orders WHERE order_id = 1003;\n\n-- merge-on-read UPDATE\nUPDATE sample_database.orders SET currency = 'GBP' WHERE customer_id = 5002;\n\n-- SELECT\nSELECT customer_id, SUM(order_amount) AS total\n FROM sample_database.orders\n GROUP BY 1\n ORDER BY 2 DESC;\n```\n\n## Two footguns the construct prevents\n\n### Footgun #1 — schema under `storageDescriptor.columns`\n\nThe CREATE succeeds but the first UPDATE silently strips\n`table_type=ICEBERG` from the table's Glue parameters, and Athena\nqueries after that fail with `HIVE_UNSUPPORTED_FORMAT`.\n\n```typescript\n// DON'T DO THIS — what most StackOverflow / re:Post examples show\nnew CfnTable(this, 'OrdersBad', {\n catalogId: this.account,\n databaseName: 'analytics',\n tableInput: {\n name: 'orders',\n tableType: 'EXTERNAL_TABLE',\n parameters: {\n table_type: 'ICEBERG',\n },\n storageDescriptor: {\n location: 's3://.../orders/',\n columns: [\n /* ... */\n ],\n },\n },\n openTableFormatInput: {\n icebergInput: {\n metadataOperation: 'CREATE',\n version: '2',\n },\n },\n});\n```\n\n`IcebergTable` instead always emits schema/partitions/sort/properties\nunder `openTableFormatInput.icebergInput.icebergTableInput`, never\nunder `storageDescriptor`.\n\n### Footgun #2 — `tableInput` co-present with `openTableFormatInput`\n\nEven setting just `tableInput: { name: 'foo' }` next to\n`openTableFormatInput` returns\n`\"Table metadata is expected only via TableInput or via IcebergTableInputProperties inside OpenTableFormatInput\"`.\nThe construct never emits `tableInput`; the table-level comment goes\ninto `tableProperties['comment']`, which lives inside\n`icebergTableInput.properties`.\n\n(There is a third footgun, field-id reuse after a column drop, that\nthe construct does **not** prevent. See the next section.)\n\n## Known limitations\n\n- **Field-id reuse is not detected across deploys.** If you drop a column with `id = 5` and then add a different column with `id = 5` in a later deploy, Glue accepts the UPDATE and Iceberg's metadata silently violates the \"never reuse a retired id\" invariant. Readers projecting old snapshots will surface deleted data under the new field's name. The construct enforces uniqueness **within one deploy** (`duplicate column id N` validator), but it doesn't compare against the live table state. The safe workflow is to always pin `id` explicitly and treat dropped ids as retired forever; never let CDK reassign an id that has ever been used.\n- **Partition field ids are positional and not pinnable.** The construct allocates partition `fieldId` densely from 1000 in the order partitions appear in `partitionSpec`. Reordering the array across deploys reassigns those ids for unchanged logical partitions, which is the partition-spec analog of the column-id-reuse footgun above. There is no `IcebergPartitionField.fieldId` pinning prop today. The safe workflow is append-only: add new partition fields at the end of `partitionSpec`, and only drop the trailing ones.\n- **CREATE-only metadata operation.** The CFN `IcebergInput.metadataOperation` only accepts `CREATE`; the construct always emits that. Subsequent deploys use Glue's normal `UpdateTable` path, which writes new Iceberg metadata in-place.\n- **Format version is immutable after CREATE.** The `formatVersion` prop is read once at table creation; changing it later requires a destroy + recreate.\n- **`merge-on-read` requires v2.** The construct rejects `write.{delete,update,merge}.mode = merge-on-read` on a v1 table at synth time.\n- **Athena DDL features that don't surface through CFN** (e.g. `ALTER TABLE WRITE ORDERED BY`, `ALTER TABLE … SET LOCATION`, `bucketed_by` / `bucket_count` Hive clauses) are not exposed. Use `IcebergPartitionTransform.bucket(N)` instead of Hive bucketing.\n- **Dropping a partition column requires a `void` intermediate per the Iceberg spec**, and the CFN `OpenTableFormatInput` cannot express that. The construct accepts the change, but Athena queries against the result will fail with `Type cannot be null`. The safe pattern is to drop partitions that source from a column while keeping that column in the schema, and only drop a column once it is no longer partitioning anything.\n\n## FAQ\n\n### How do I create an Apache Iceberg table in AWS CDK?\n\nInstall `cdk-glue-iceberg-table`, then declare an `IcebergTable` with a\ndatabase, a column list, and an S3 `location` — see [Use](#use).\n`cdk deploy` creates the Glue Data Catalog table and writes the\nIceberg `metadata.json` under your S3 prefix. No custom resource or\nLambda is involved; it is a plain `AWS::Glue::Table` in your\nCloudFormation template.\n\n### How do I evolve an Iceberg table schema (add, rename, or drop a column) in CloudFormation?\n\nChange the `columns` array and run `cdk deploy` again. The construct\npasses the new schema to Glue's `UpdateTable`, which writes a new\n`metadata.json` with a new `schema-id`; existing data files stay\nreadable because each column's `id` is pinned and never reused. The\nsame applies to `partitionSpec`. See [Evolving schema and\npartitions](#evolving-schema-and-partitions).\n\n### Does CloudFormation support Iceberg tables natively?\n\nYes, through `AWS::Glue::Table` with `OpenTableFormatInput.IcebergInput`,\ndocumented by AWS in [December 2025](https://aws.amazon.com/blogs/big-data/create-and-update-apache-iceberg-tables-with-partitions-in-the-aws-glue-data-catalog-using-the-aws-sdk-and-aws-cloudformation).\nThe catch is that the raw shape corrupts the table on the first\n`Update` if you place schema under `storageDescriptor.columns` or set\n`tableInput` alongside `openTableFormatInput`. This construct only ever\nemits the safe shape and gives you no way to express the unsafe ones.\nSee [Two footguns the construct prevents](#two-footguns-the-construct-prevents).\n\n### What is the difference between cdk-glue-iceberg-table and a raw CfnTable?\n\nA raw `CfnTable` makes you hand-write the `OpenTableFormatInput` JSON\nand own both silent-corruption footguns. `cdk-glue-iceberg-table` gives\nyou a typed `IcebergType` / `IcebergPartitionTransform` API,\nsynth-time validation (partition transforms checked against column\ntypes, format/version mismatches caught before deploy), pinned field\nIDs for safe evolution, and `grantRead` / `grantWrite` helpers. See\n[How it compares](#how-it-compares).\n\n### How do I create a partitioned Iceberg table in CDK?\n\nPass a `partitionSpec` of `IcebergPartitionTransform` entries —\n`identity`, `bucket(N)`, `truncate(W)`, `year`, `month`, `day`, `hour`,\nor `void`. Each transform is validated against its source column's type\nat synth time. See [Using `IcebergTable`](#using-icebergtable).\n\n### Does it work with Athena and Lake Formation?\n\nYes. The demo app in the [`arceus`](https://github.com/ksco92/arceus)\nrepo registers the tables with Lake Formation and queries them from\nAthena, including v2 merge-on-read `INSERT` / `UPDATE` / `DELETE` /\n`MERGE`, time travel, `OPTIMIZE`, and `VACUUM`. The construct's\n`grant*` helpers issue IAM grants; under Lake Formation you still add\nthe matching `SELECT` / `INSERT` LF grants. See [Granting\naccess](#granting-access).\n\n### Can I use Iceberg v2 merge-on-read (row-level UPDATE and DELETE)?\n\nYes — set `formatVersion: IcebergFormatVersion.V2` and the\n`write.{delete,update,merge}.mode = merge-on-read` table properties.\nThe construct rejects merge-on-read on a v1 table at synth time.\n\n## Contributing\n\nDevelopment, the monorepo layout, and the demo app live in [CONTRIBUTING.md](https://github.com/ksco92/arceus/blob/main/CONTRIBUTING.md).\n\n## License\n\n[MIT](LICENSE).\n"
|
|
9073
|
+
"markdown": "# cdk-glue-iceberg-table\n\n[](https://github.com/ksco92/arceus/actions/workflows/ci.yml)\n[](https://codecov.io/gh/ksco92/arceus)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://pypi.org/project/cdk-glue-iceberg-table/)\n[](https://constructs.dev/packages/cdk-glue-iceberg-table)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://www.npmjs.com/package/cdk-glue-iceberg-table)\n[](https://github.com/ksco92/arceus/commits/main)\n[](LICENSE)\n\n**`cdk-glue-iceberg-table` is the AWS CDK L2 construct for creating\nand evolving Apache Iceberg tables in the AWS Glue Data Catalog.** It\nemits the exact `AWS::Glue::Table` / `OpenTableFormatInput` shape that\nsurvives a CloudFormation `Update`, so a single `cdk deploy` creates a\ntable, evolves its schema and partitions, and destroys it, the same\nlifecycle CDK gives any other resource. No custom resource, no Lambda,\nno \"`cdk deploy` and then run this SQL by hand\" two-step. Table\nchanges (new columns, renames, drops, new partition fields) land as\na reviewed diff in a pull request and apply through `cdk deploy`.\n\nStatus (June 2026): published on npm (TypeScript/JavaScript) and PyPI\n(Python), pre-1.0, with a public surface guarded by an end-to-end\nconsumer test on every PR plus three real-AWS integration suites. The\nmulti-language packages are generated from one TypeScript source with\n[jsii](https://aws.github.io/jsii/) and indexed on\n[Construct Hub](https://constructs.dev/packages/cdk-glue-iceberg-table).\nDeveloped in the\n[`ksco92/arceus`](https://github.com/ksco92/arceus) monorepo, which\nalso holds a CDK demo app that dogfoods the construct against a real\nAWS account.\n\n### Why this exists\n\nAWS documents the CloudFormation shape for Iceberg tables ([AWS Big\nData blog, December 2025](https://aws.amazon.com/blogs/big-data/create-and-update-apache-iceberg-tables-with-partitions-in-the-aws-glue-data-catalog-using-the-aws-sdk-and-aws-cloudformation)),\nbut raw `AWS::Glue::Table` is a minefield: the CREATE succeeds and the\n*first* `Update` silently strips `table_type=ICEBERG`, after which\nevery Athena query fails with `HIVE_UNSUPPORTED_FORMAT`. The\nmotivating CDK issue is [aws/aws-cdk#29660](https://github.com/aws/aws-cdk/issues/29660);\n[`manmartgarc`'s comment](https://github.com/aws/aws-cdk/issues/29660#issuecomment-4411359248)\ndocuments the only working shape and the silent-corruption traps you\nhit by getting it slightly wrong. This construct implements that\nshape, refuses to emit the unsafe alternatives, and pins Iceberg field\nIDs so schema evolution never orphans existing data. It is the basis\nof the upstream proposal to land an `IcebergTable` in\n`@aws-cdk/aws-glue-alpha` ([aws/aws-cdk#37988](https://github.com/aws/aws-cdk/pull/37988));\nthis package tracks that proposal and stays current with it.\n\n### How it compares\n\n| Approach | Declarative IaC (in the PR) | In-place schema + partition evolution | Prevents the silent-corruption footguns | Typed API + synth-time validation |\n| --- | --- | --- | --- | --- |\n| Raw `CfnTable` (L1) | ✅ | ⚠️ only if you hand-write the exact `OpenTableFormatInput` shape | ❌ you own both footguns | ❌ |\n| CDK custom resource (Lambda + Glue SDK) | ⚠️ via a custom resource you maintain | ⚠️ you write the diff logic | ⚠️ your code's responsibility | ❌ |\n| Spark / Athena SQL DDL at job runtime | ❌ imperative, outside the PR | ✅ but outside CloudFormation | n/a | ❌ |\n| **`cdk-glue-iceberg-table`** | ✅ | ✅ via `cdk deploy` | ✅ both prevented by construction | ✅ |\n\n## Install\n\nTypeScript / JavaScript (npm):\n\n```bash\nnpm install cdk-glue-iceberg-table\n```\n\nPython (PyPI):\n\n```bash\npip install cdk-glue-iceberg-table\n```\n\nPeer dependencies (your CDK app must already have these). For\nTypeScript / JavaScript:\n\n```bash\nnpm install aws-cdk-lib constructs @aws-cdk/aws-glue-alpha\n```\n\nFor Python:\n\n```bash\npip install aws-cdk-lib constructs aws-cdk.aws-glue-alpha\n```\n\n## Use\n\n### TypeScript\n\n```ts\nimport { Bucket } from 'aws-cdk-lib/aws-s3';\nimport { Database } from '@aws-cdk/aws-glue-alpha';\nimport {\n IcebergTable,\n IcebergType,\n IcebergPartitionTransform,\n} from 'cdk-glue-iceberg-table';\n\nconst bucket = new Bucket(this, 'Warehouse');\nconst db = new Database(this, 'Db', { databaseName: 'analytics' });\n\nnew IcebergTable(this, 'OrdersTable', {\n database: db,\n tableName: 'orders',\n location: `s3://${bucket.bucketName}/analytics/orders/`,\n columns: [\n { name: 'order_id', type: IcebergType.LONG, required: true, id: 1 },\n { name: 'customer_id', type: IcebergType.LONG, required: true, id: 2 },\n { name: 'placed_at', type: IcebergType.TIMESTAMPTZ, required: true, id: 3 },\n ],\n partitionSpec: [\n { sourceColumn: 'placed_at', transform: IcebergPartitionTransform.DAY, fieldId: 1000 },\n { sourceColumn: 'customer_id', transform: IcebergPartitionTransform.bucket(16), fieldId: 1001 },\n ],\n identifierFieldNames: ['order_id'],\n});\n```\n\n### Python\n\nThe same table in Python — the API mirrors TypeScript with\n`snake_case` props and `PascalCase` types:\n\n```python\nfrom aws_cdk.aws_s3 import Bucket\nfrom aws_cdk.aws_glue_alpha import Database\nfrom cdk_glue_iceberg_table import (\n IcebergTable,\n IcebergType,\n IcebergPartitionTransform,\n)\n\nbucket = Bucket(self, \"Warehouse\")\ndb = Database(self, \"Db\", database_name=\"analytics\")\n\nIcebergTable(self, \"OrdersTable\",\n database=db,\n table_name=\"orders\",\n location=f\"s3://{bucket.bucket_name}/analytics/orders/\",\n columns=[\n {\"name\": \"order_id\", \"type\": IcebergType.LONG, \"required\": True, \"id\": 1},\n {\"name\": \"customer_id\", \"type\": IcebergType.LONG, \"required\": True, \"id\": 2},\n {\"name\": \"placed_at\", \"type\": IcebergType.TIMESTAMPTZ, \"required\": True, \"id\": 3},\n ],\n partition_spec=[\n {\"source_column\": \"placed_at\", \"transform\": IcebergPartitionTransform.DAY, \"field_id\": 1000},\n {\"source_column\": \"customer_id\", \"transform\": IcebergPartitionTransform.bucket(16), \"field_id\": 1001},\n ],\n identifier_field_names=[\"order_id\"],\n)\n```\n\nConsumer-facing reference sections:\n\n- [How it compares](#how-it-compares) — `cdk-glue-iceberg-table` vs raw `CfnTable`, custom resources, and runtime SQL DDL.\n- [Using `IcebergTable`](#using-icebergtable) — full API reference with examples.\n- [Two footguns the construct prevents](#two-footguns-the-construct-prevents) — the silent-corruption traps that motivated this construct.\n- [Known limitations](#known-limitations) — what the construct does and doesn't enforce.\n- [FAQ](#faq) — common \"how do I … in CDK / CloudFormation\" questions.\n\n## Exported surface\n\nThe package entry point re-exports everything you import from\n`cdk-glue-iceberg-table`. In TypeScript / JavaScript the compiled\nsurface lives under `dist/lib/iceberg/` after `npm install`; in Python\nthe same surface is imported from the `cdk_glue_iceberg_table` module\nafter `pip install`:\n\n- **`IcebergTable`** — the L2 construct itself, with `grantRead` / `grantWrite` / `grantReadWrite` and the `fromIcebergTableAttributes(...)` import factory.\n- **`IcebergType`** — primitive types plus `list` / `map` / `struct` / `decimal` / `fixed` factories. Renders to the JSON shape Glue's `IcebergStructField.type` expects.\n- **`IcebergPartitionTransform`** — identity / `bucket(N)` / `truncate(W)` / year / month / day / hour / void. Each transform validates against the source column type at synth time.\n- **`IcebergDataFormat`** (parquet/orc/avro, default parquet), **`IcebergFormatVersion`** (v1/v2, required — set explicitly per table), **`IcebergSortDirection`**, **`IcebergNullOrder`**, plus a `tableProperties` validator that catches misconfigured properties before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).\n\n## Using `IcebergTable`\n\nA minimal table:\n\n```typescript\nimport {\n Database,\n} from '@aws-cdk/aws-glue-alpha';\nimport {\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nconst db = new Database(this, 'Db', {\n databaseName: 'analytics',\n});\n\nnew IcebergTable(this, 'Users', {\n database: db,\n tableName: 'users',\n columns: [\n {\n name: 'user_id',\n type: IcebergType.LONG,\n required: true,\n id: 1,\n },\n {\n name: 'email',\n type: IcebergType.STRING,\n required: true,\n id: 2,\n },\n {\n name: 'signed_up_at',\n type: IcebergType.TIMESTAMPTZ,\n required: true,\n id: 3,\n },\n ],\n location: `s3://${bucket.bucketName}/analytics/users/`,\n});\n```\n\nEvery top-level column must declare an `id`: a stable, unique,\npositive integer you pick once and never reuse or reassign. Because\nIceberg data files reference fields by id rather than by name, a fixed\nid is what lets you add, rename, and drop columns across deploys\nwithout corrupting old data — see\n[How do I evolve an Iceberg table schema](#how-do-i-evolve-an-iceberg-table-schema-add-rename-or-drop-a-column-in-cloudformation).\nOnce a column is dropped, leave its id retired; never hand it to a new\ncolumn. (Nested struct/list/map field ids are assigned automatically\nabove the highest top-level id — only top-level column ids are\ncaller-provided.)\n\nA table that exercises most of the surface (partitions, sort order,\nnested types, identifier fields, table properties, removal policy):\n\n```typescript\nimport {\n RemovalPolicy,\n} from 'aws-cdk-lib';\nimport {\n Database,\n} from '@aws-cdk/aws-glue-alpha';\nimport {\n IcebergDataFormat,\n IcebergFormatVersion,\n IcebergNullOrder,\n IcebergPartitionTransform,\n IcebergSortDirection,\n IcebergTable,\n IcebergType,\n} from 'cdk-glue-iceberg-table';\n\nnew IcebergTable(this, 'OrdersTable', {\n database: db,\n tableName: 'orders',\n comment: 'Demo Iceberg orders table — exercises partitions, sort order, and merge-on-read.',\n columns: [\n {\n name: 'order_id',\n type: IcebergType.LONG,\n required: true,\n id: 1,\n },\n {\n name: 'customer_id',\n type: IcebergType.LONG,\n required: true,\n id: 2,\n },\n {\n name: 'order_amount',\n type: IcebergType.decimal(12, 2),\n required: true,\n id: 3,\n },\n {\n name: 'currency',\n type: IcebergType.STRING,\n required: true,\n id: 4,\n },\n {\n name: 'placed_at',\n type: IcebergType.TIMESTAMPTZ,\n required: true,\n id: 5,\n },\n {\n name: 'tags',\n type: IcebergType.list(IcebergType.STRING),\n id: 6,\n },\n {\n name: 'shipping_address',\n type: IcebergType.struct([\n {\n name: 'line1',\n type: IcebergType.STRING,\n required: true,\n },\n {\n name: 'city',\n type: IcebergType.STRING,\n required: true,\n },\n {\n name: 'country',\n type: IcebergType.STRING,\n required: true,\n },\n {\n name: 'postal_code',\n type: IcebergType.STRING,\n },\n ]),\n id: 7,\n },\n {\n name: 'metadata',\n type: IcebergType.map(IcebergType.STRING, IcebergType.STRING, false),\n id: 8,\n },\n ],\n location: `s3://${bucket.bucketName}/analytics/orders/`,\n partitionSpec: [\n {\n sourceColumn: 'placed_at',\n transform: IcebergPartitionTransform.DAY,\n fieldId: 1000,\n },\n {\n sourceColumn: 'customer_id',\n transform: IcebergPartitionTransform.bucket(16),\n fieldId: 1001,\n },\n ],\n sortOrder: [\n {\n sourceColumn: 'placed_at',\n direction: IcebergSortDirection.ASC,\n nullOrder: IcebergNullOrder.NULLS_LAST,\n },\n {\n sourceColumn: 'order_id',\n direction: IcebergSortDirection.ASC,\n },\n ],\n identifierFieldNames: [\n 'order_id',\n ],\n dataFormat: IcebergDataFormat.PARQUET,\n formatVersion: IcebergFormatVersion.V2,\n tableProperties: {\n 'write.parquet.compression-codec': 'zstd',\n 'write.delete.mode': 'merge-on-read',\n 'write.update.mode': 'merge-on-read',\n 'write.merge.mode': 'merge-on-read',\n 'write.target-file-size-bytes': '134217728',\n 'history.expire.min-snapshots-to-keep': '5',\n 'gc.enabled': 'true',\n },\n removalPolicy: RemovalPolicy.DESTROY,\n});\n```\n\nEach partition field also pins a `fieldId`: an integer of 1000 or\nabove (the Iceberg spec reserves lower ids for schema columns), unique\nacross the table's entire partition-spec history. The discipline is\nthe inverse of the column-id rule. A column keeps its id forever; a\npartition field keeps its id only while it stays untouched. Change\nanything about a field — swap `month` for `day`, point it at a\ndifferent source column — and it becomes a *new* field that needs a\nfresh, never-before-used id, while the old id retires with the old\nspec. Reusing a retired id (or reassigning an id to a different field)\nproduces a table that Athena still reads but Spark and Trino reject\nwith `ValidationException: Conflicting partition fields`.\n\nThe resulting Iceberg `metadata.json` for this table contains every\nfeature you set:\n\n```json\n{\n \"format-version\": 2,\n \"table-uuid\": \"39a948f9-...\",\n \"current-schema-id\": 0,\n \"schemas\": [\n {\n \"schema-id\": 0,\n \"identifier-field-ids\": [1],\n \"fields\": [\n { \"id\": 1, \"name\": \"order_id\", \"required\": true, \"type\": \"long\" },\n { \"id\": 2, \"name\": \"customer_id\", \"required\": true, \"type\": \"long\" },\n { \"id\": 3, \"name\": \"order_amount\", \"required\": true, \"type\": \"decimal(12, 2)\" },\n { \"id\": 4, \"name\": \"currency\", \"required\": true, \"type\": \"string\" },\n { \"id\": 5, \"name\": \"placed_at\", \"required\": true, \"type\": \"timestamptz\" },\n { \"id\": 6, \"name\": \"tags\", \"required\": false,\n \"type\": { \"type\": \"list\", \"element-id\": 9, \"element\": \"string\", \"element-required\": true } },\n { \"id\": 7, \"name\": \"shipping_address\", \"required\": false,\n \"type\": { \"type\": \"struct\", \"fields\": [\n { \"id\": 10, \"name\": \"line1\", \"required\": true, \"type\": \"string\" },\n { \"id\": 11, \"name\": \"city\", \"required\": true, \"type\": \"string\" },\n { \"id\": 12, \"name\": \"country\", \"required\": true, \"type\": \"string\" },\n { \"id\": 13, \"name\": \"postal_code\", \"required\": false, \"type\": \"string\" }\n ] } },\n { \"id\": 8, \"name\": \"metadata\", \"required\": false,\n \"type\": { \"type\": \"map\", \"key-id\": 14, \"key\": \"string\", \"value-id\": 15,\n \"value-required\": false, \"value\": \"string\" } }\n ]\n }\n ],\n \"partition-specs\": [\n { \"spec-id\": 0, \"fields\": [\n { \"name\": \"placed_at_day\", \"transform\": \"day\", \"source-id\": 5, \"field-id\": 1000 },\n { \"name\": \"customer_id_bucket\", \"transform\": \"bucket[16]\", \"source-id\": 2, \"field-id\": 1001 }\n ]}\n ],\n \"sort-orders\": [\n { \"order-id\": 1, \"fields\": [\n { \"transform\": \"identity\", \"source-id\": 5, \"direction\": \"asc\", \"null-order\": \"nulls-last\" },\n { \"transform\": \"identity\", \"source-id\": 1, \"direction\": \"asc\", \"null-order\": \"nulls-last\" }\n ]}\n ],\n \"properties\": {\n \"format-version\": \"2\",\n \"write.format.default\": \"parquet\",\n \"write.parquet.compression-codec\": \"zstd\",\n \"write.merge.mode\": \"merge-on-read\",\n \"write.update.mode\": \"merge-on-read\",\n \"write.delete.mode\": \"merge-on-read\",\n \"write.target-file-size-bytes\": \"134217728\",\n \"history.expire.min-snapshots-to-keep\": \"5\",\n \"gc.enabled\": \"true\",\n \"comment\": \"Demo Iceberg orders table — exercises partitions, sort order, and merge-on-read.\"\n }\n}\n```\n\n### Granting access\n\n```typescript\ntable.grantRead(role); // Glue read + S3 read on the table's prefix\ntable.grantWrite(role); // Glue write + S3 write\ntable.grantReadWrite(role);\n```\n\nThe `grant*` helpers issue **IAM grants only**. Under Lake Formation\nyou still add the matching `SELECT` / `INSERT` / `DELETE` LF grants on\ntop of the construct's IAM grants for Athena queries to succeed.\n\n### Importing an existing table\n\n```typescript\nconst existing = IcebergTable.fromIcebergTableAttributes(this, 'Orders', {\n database: db,\n tableName: 'orders',\n location: 's3://my-bucket/analytics/orders/',\n});\nexisting.grantRead(role);\n```\n\n### Evolving schema and partitions\n\nChange the `columns` array (or `partitionSpec`) and run `cdk deploy`\nagain. The construct passes the new schema to Glue's `UpdateTable`,\nwhich writes a new `metadata.json` with a new `schema-id`; existing\ndata files stay readable because each column's `id` is pinned and\nnever reused. Adds, renames (same `id`, new `name`), and drops all\nflow through `cdk deploy` alone — no out-of-band SQL DDL.\n\nPartition evolution flows the same way, with one extra rule on\n`fieldId`: a changed partition field is a new partition field. To move\na table from `month(placed_at)` to `day(placed_at)`, drop the month\nentry and add a day entry under a fresh `fieldId` — never recycle the\nmonth field's id:\n\n```typescript\npartitionSpec: [\n // was: { sourceColumn: 'placed_at', transform: IcebergPartitionTransform.MONTH, fieldId: 1000 },\n {\n sourceColumn: 'placed_at',\n transform: IcebergPartitionTransform.DAY,\n fieldId: 1001,\n },\n],\n```\n\nOld data files written under the month spec stay readable; new writes\npartition by day. Reusing `1000` for the day field instead corrupts\nthe spec history for strict engines (Spark, Trino) while Athena keeps\nworking — the failure only surfaces at Spark runtime.\n\n### Inserting and querying\n\n```sql\n-- INSERT into the orders table\nINSERT INTO sample_database.orders VALUES\n (1001, 5001, DECIMAL '149.99', 'USD',\n TIMESTAMP '2026-05-20 09:15:00 UTC',\n ARRAY['holiday-promo','first-order'],\n CAST(ROW('1 Infinite Loop','Cupertino','US','95014')\n AS ROW(line1 VARCHAR,city VARCHAR,country VARCHAR,postal_code VARCHAR)),\n MAP(ARRAY['channel','utm'], ARRAY['web','google'])),\n -- ... more rows\n;\n\n-- merge-on-read DELETE (only legal because we chose v2 + merge-on-read mode)\nDELETE FROM sample_database.orders WHERE order_id = 1003;\n\n-- merge-on-read UPDATE\nUPDATE sample_database.orders SET currency = 'GBP' WHERE customer_id = 5002;\n\n-- SELECT\nSELECT customer_id, SUM(order_amount) AS total\n FROM sample_database.orders\n GROUP BY 1\n ORDER BY 2 DESC;\n```\n\n## Two footguns the construct prevents\n\n### Footgun #1 — schema under `storageDescriptor.columns`\n\nThe CREATE succeeds but the first UPDATE silently strips\n`table_type=ICEBERG` from the table's Glue parameters, and Athena\nqueries after that fail with `HIVE_UNSUPPORTED_FORMAT`.\n\n```typescript\n// DON'T DO THIS — what most StackOverflow / re:Post examples show\nnew CfnTable(this, 'OrdersBad', {\n catalogId: this.account,\n databaseName: 'analytics',\n tableInput: {\n name: 'orders',\n tableType: 'EXTERNAL_TABLE',\n parameters: {\n table_type: 'ICEBERG',\n },\n storageDescriptor: {\n location: 's3://.../orders/',\n columns: [\n /* ... */\n ],\n },\n },\n openTableFormatInput: {\n icebergInput: {\n metadataOperation: 'CREATE',\n version: '2',\n },\n },\n});\n```\n\n`IcebergTable` instead always emits schema/partitions/sort/properties\nunder `openTableFormatInput.icebergInput.icebergTableInput`, never\nunder `storageDescriptor`.\n\n### Footgun #2 — `tableInput` co-present with `openTableFormatInput`\n\nEven setting just `tableInput: { name: 'foo' }` next to\n`openTableFormatInput` returns\n`\"Table metadata is expected only via TableInput or via IcebergTableInputProperties inside OpenTableFormatInput\"`.\nThe construct never emits `tableInput`; the table-level comment goes\ninto `tableProperties['comment']`, which lives inside\n`icebergTableInput.properties`.\n\n(There is a third footgun, field-id reuse after a column drop, that\nthe construct does **not** prevent. See the next section.)\n\n## Known limitations\n\n- **Field-id reuse is not detected across deploys.** If you drop a column with `id = 5` and then add a different column with `id = 5` in a later deploy, Glue accepts the UPDATE and Iceberg's metadata silently violates the \"never reuse a retired id\" invariant. Readers projecting old snapshots will surface deleted data under the new field's name. The construct enforces uniqueness **within one deploy** (`duplicate column id N` validator), but it doesn't compare against the live table state. The safe workflow is to always pin `id` explicitly and treat dropped ids as retired forever; never let CDK reassign an id that has ever been used.\n- **Partition field-id reuse is not detected across deploys either.** The same live-state blindness applies to partition `fieldId`s: the construct validates them within one deploy (integer >= 1000, unique in the spec), but it cannot tell that the id you pinned already appeared in an earlier spec of the deployed table. The invariant partition ids must uphold is stricter than the column one — unique across the table's **entire spec history**, so a changed field (different transform or source column) must take a fresh id rather than keep its old one. Violate it and strict engines (Spark, Trino) reject the table with `Conflicting partition fields` while Athena keeps answering, masking the break until Spark runtime. The safe workflow: treat every id that has ever appeared in any spec as retired, and allocate new fields the next never-used id.\n- **`last-partition-id` is written by Glue, not by the construct.** The construct supplies the spec's field ids; Glue's `UpdateTable` computes the table's `last-partition-id` counter when it writes the new metadata. The repo's integration test asserts the counter keeps up with the highest supplied field id after a partition evolution, but engines that later evolve the spec themselves (e.g. Spark `ALTER TABLE … ADD PARTITION FIELD`) allocate from that counter — if you mix CDK-managed and engine-managed partition evolution, verify the counter in `metadata.json` before trusting engine-side changes.\n- **CREATE-only metadata operation.** The CFN `IcebergInput.metadataOperation` only accepts `CREATE`; the construct always emits that. Subsequent deploys use Glue's normal `UpdateTable` path, which writes new Iceberg metadata in-place.\n- **Format version is immutable after CREATE.** The `formatVersion` prop is read once at table creation; changing it later requires a destroy + recreate.\n- **`merge-on-read` requires v2.** The construct rejects `write.{delete,update,merge}.mode = merge-on-read` on a v1 table at synth time.\n- **Athena DDL features that don't surface through CFN** (e.g. `ALTER TABLE WRITE ORDERED BY`, `ALTER TABLE … SET LOCATION`, `bucketed_by` / `bucket_count` Hive clauses) are not exposed. Use `IcebergPartitionTransform.bucket(N)` instead of Hive bucketing.\n- **Dropping a partition column requires a `void` intermediate per the Iceberg spec**, and the CFN `OpenTableFormatInput` cannot express that. The construct accepts the change, but Athena queries against the result will fail with `Type cannot be null`. The safe pattern is to drop partitions that source from a column while keeping that column in the schema, and only drop a column once it is no longer partitioning anything.\n\n## FAQ\n\n### How do I create an Apache Iceberg table in AWS CDK?\n\nInstall `cdk-glue-iceberg-table`, then declare an `IcebergTable` with a\ndatabase, a column list, and an S3 `location` — see [Use](#use).\n`cdk deploy` creates the Glue Data Catalog table and writes the\nIceberg `metadata.json` under your S3 prefix. No custom resource or\nLambda is involved; it is a plain `AWS::Glue::Table` in your\nCloudFormation template.\n\n### How do I evolve an Iceberg table schema (add, rename, or drop a column) in CloudFormation?\n\nChange the `columns` array and run `cdk deploy` again. The construct\npasses the new schema to Glue's `UpdateTable`, which writes a new\n`metadata.json` with a new `schema-id`; existing data files stay\nreadable because each column's `id` is pinned and never reused.\nPartition changes work the same way, except a changed partition field\ntakes a fresh `fieldId` instead of keeping its old one. See [Evolving\nschema and partitions](#evolving-schema-and-partitions).\n\n### Does CloudFormation support Iceberg tables natively?\n\nYes, through `AWS::Glue::Table` with `OpenTableFormatInput.IcebergInput`,\ndocumented by AWS in [December 2025](https://aws.amazon.com/blogs/big-data/create-and-update-apache-iceberg-tables-with-partitions-in-the-aws-glue-data-catalog-using-the-aws-sdk-and-aws-cloudformation).\nThe catch is that the raw shape corrupts the table on the first\n`Update` if you place schema under `storageDescriptor.columns` or set\n`tableInput` alongside `openTableFormatInput`. This construct only ever\nemits the safe shape and gives you no way to express the unsafe ones.\nSee [Two footguns the construct prevents](#two-footguns-the-construct-prevents).\n\n### What is the difference between cdk-glue-iceberg-table and a raw CfnTable?\n\nA raw `CfnTable` makes you hand-write the `OpenTableFormatInput` JSON\nand own both silent-corruption footguns. `cdk-glue-iceberg-table` gives\nyou a typed `IcebergType` / `IcebergPartitionTransform` API,\nsynth-time validation (partition transforms checked against column\ntypes, format/version mismatches caught before deploy), pinned field\nIDs for safe evolution, and `grantRead` / `grantWrite` helpers. See\n[How it compares](#how-it-compares).\n\n### How do I create a partitioned Iceberg table in CDK?\n\nPass a `partitionSpec` of `IcebergPartitionTransform` entries —\n`identity`, `bucket(N)`, `truncate(W)`, `year`, `month`, `day`, `hour`,\nor `void` — each pinned to a `fieldId` of 1000 or above. Transforms are\nvalidated against their source column's type at synth time. See\n[Using `IcebergTable`](#using-icebergtable).\n\n### Does it work with Athena and Lake Formation?\n\nYes. The demo app in the [`arceus`](https://github.com/ksco92/arceus)\nrepo registers the tables with Lake Formation and queries them from\nAthena, including v2 merge-on-read `INSERT` / `UPDATE` / `DELETE` /\n`MERGE`, time travel, `OPTIMIZE`, and `VACUUM`. The construct's\n`grant*` helpers issue IAM grants; under Lake Formation you still add\nthe matching `SELECT` / `INSERT` LF grants. See [Granting\naccess](#granting-access).\n\n### Can I use Iceberg v2 merge-on-read (row-level UPDATE and DELETE)?\n\nYes — set `formatVersion: IcebergFormatVersion.V2` and the\n`write.{delete,update,merge}.mode = merge-on-read` table properties.\nThe construct rejects merge-on-read on a v1 table at synth time.\n\n## Contributing\n\nDevelopment, the monorepo layout, and the demo app live in [CONTRIBUTING.md](https://github.com/ksco92/arceus/blob/main/CONTRIBUTING.md).\n\n## License\n\n[MIT](LICENSE).\n"
|
|
9074
9074
|
},
|
|
9075
9075
|
"repository": {
|
|
9076
9076
|
"directory": "packages/cdk-glue-iceberg-table",
|
|
@@ -9098,7 +9098,7 @@
|
|
|
9098
9098
|
"kind": "interface",
|
|
9099
9099
|
"locationInModule": {
|
|
9100
9100
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9101
|
-
"line":
|
|
9101
|
+
"line": 222
|
|
9102
9102
|
},
|
|
9103
9103
|
"methods": [
|
|
9104
9104
|
{
|
|
@@ -9109,7 +9109,7 @@
|
|
|
9109
9109
|
},
|
|
9110
9110
|
"locationInModule": {
|
|
9111
9111
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9112
|
-
"line":
|
|
9112
|
+
"line": 236
|
|
9113
9113
|
},
|
|
9114
9114
|
"name": "grantRead",
|
|
9115
9115
|
"parameters": [
|
|
@@ -9134,7 +9134,7 @@
|
|
|
9134
9134
|
},
|
|
9135
9135
|
"locationInModule": {
|
|
9136
9136
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9137
|
-
"line":
|
|
9137
|
+
"line": 242
|
|
9138
9138
|
},
|
|
9139
9139
|
"name": "grantReadWrite",
|
|
9140
9140
|
"parameters": [
|
|
@@ -9159,7 +9159,7 @@
|
|
|
9159
9159
|
},
|
|
9160
9160
|
"locationInModule": {
|
|
9161
9161
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9162
|
-
"line":
|
|
9162
|
+
"line": 239
|
|
9163
9163
|
},
|
|
9164
9164
|
"name": "grantWrite",
|
|
9165
9165
|
"parameters": [
|
|
@@ -9188,7 +9188,7 @@
|
|
|
9188
9188
|
"immutable": true,
|
|
9189
9189
|
"locationInModule": {
|
|
9190
9190
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9191
|
-
"line":
|
|
9191
|
+
"line": 230
|
|
9192
9192
|
},
|
|
9193
9193
|
"name": "database",
|
|
9194
9194
|
"type": {
|
|
@@ -9204,7 +9204,7 @@
|
|
|
9204
9204
|
"immutable": true,
|
|
9205
9205
|
"locationInModule": {
|
|
9206
9206
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9207
|
-
"line":
|
|
9207
|
+
"line": 233
|
|
9208
9208
|
},
|
|
9209
9209
|
"name": "location",
|
|
9210
9210
|
"type": {
|
|
@@ -9220,7 +9220,7 @@
|
|
|
9220
9220
|
"immutable": true,
|
|
9221
9221
|
"locationInModule": {
|
|
9222
9222
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9223
|
-
"line":
|
|
9223
|
+
"line": 224
|
|
9224
9224
|
},
|
|
9225
9225
|
"name": "tableArn",
|
|
9226
9226
|
"type": {
|
|
@@ -9236,7 +9236,7 @@
|
|
|
9236
9236
|
"immutable": true,
|
|
9237
9237
|
"locationInModule": {
|
|
9238
9238
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9239
|
-
"line":
|
|
9239
|
+
"line": 227
|
|
9240
9240
|
},
|
|
9241
9241
|
"name": "tableName",
|
|
9242
9242
|
"type": {
|
|
@@ -9464,6 +9464,23 @@
|
|
|
9464
9464
|
},
|
|
9465
9465
|
"name": "IcebergPartitionField",
|
|
9466
9466
|
"properties": [
|
|
9467
|
+
{
|
|
9468
|
+
"abstract": true,
|
|
9469
|
+
"docs": {
|
|
9470
|
+
"remarks": "Required: every partition field must\nbe pinned to a stable integer >= 1000 (the Iceberg spec reserves\nids below 1000 for schema columns) that is unique across the\ntable's entire partition-spec history — not just the current\nspec. Iceberg's strict engines (Spark, Trino) reject a table\nwhose spec history carries two different partition fields with\nthe same id.\n\nThe rule is the inverse of the column-id rule: a column keeps\nits id forever, but any change to a partition field — a\ndifferent transform, a different source column — makes it a NEW\nfield that needs a fresh, never-before-used id. Only an\nuntouched field keeps its id across deploys. Once a field is\nchanged or dropped, its id is retired for good; the construct\ncannot see prior deploys, so honoring retirement is on you.",
|
|
9471
|
+
"stability": "experimental",
|
|
9472
|
+
"summary": "Iceberg partition field id."
|
|
9473
|
+
},
|
|
9474
|
+
"immutable": true,
|
|
9475
|
+
"locationInModule": {
|
|
9476
|
+
"filename": "lib/iceberg/iceberg-table.ts",
|
|
9477
|
+
"line": 142
|
|
9478
|
+
},
|
|
9479
|
+
"name": "fieldId",
|
|
9480
|
+
"type": {
|
|
9481
|
+
"primitive": "number"
|
|
9482
|
+
}
|
|
9483
|
+
},
|
|
9467
9484
|
{
|
|
9468
9485
|
"abstract": true,
|
|
9469
9486
|
"docs": {
|
|
@@ -10215,7 +10232,7 @@
|
|
|
10215
10232
|
"kind": "interface",
|
|
10216
10233
|
"locationInModule": {
|
|
10217
10234
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10218
|
-
"line":
|
|
10235
|
+
"line": 146
|
|
10219
10236
|
},
|
|
10220
10237
|
"name": "IcebergSortField",
|
|
10221
10238
|
"properties": [
|
|
@@ -10228,7 +10245,7 @@
|
|
|
10228
10245
|
"immutable": true,
|
|
10229
10246
|
"locationInModule": {
|
|
10230
10247
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10231
|
-
"line":
|
|
10248
|
+
"line": 148
|
|
10232
10249
|
},
|
|
10233
10250
|
"name": "sourceColumn",
|
|
10234
10251
|
"type": {
|
|
@@ -10245,7 +10262,7 @@
|
|
|
10245
10262
|
"immutable": true,
|
|
10246
10263
|
"locationInModule": {
|
|
10247
10264
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10248
|
-
"line":
|
|
10265
|
+
"line": 154
|
|
10249
10266
|
},
|
|
10250
10267
|
"name": "direction",
|
|
10251
10268
|
"optional": true,
|
|
@@ -10263,7 +10280,7 @@
|
|
|
10263
10280
|
"immutable": true,
|
|
10264
10281
|
"locationInModule": {
|
|
10265
10282
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10266
|
-
"line":
|
|
10283
|
+
"line": 157
|
|
10267
10284
|
},
|
|
10268
10285
|
"name": "nullOrder",
|
|
10269
10286
|
"optional": true,
|
|
@@ -10281,7 +10298,7 @@
|
|
|
10281
10298
|
"immutable": true,
|
|
10282
10299
|
"locationInModule": {
|
|
10283
10300
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10284
|
-
"line":
|
|
10301
|
+
"line": 151
|
|
10285
10302
|
},
|
|
10286
10303
|
"name": "transform",
|
|
10287
10304
|
"optional": true,
|
|
@@ -10392,7 +10409,7 @@
|
|
|
10392
10409
|
},
|
|
10393
10410
|
"locationInModule": {
|
|
10394
10411
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10395
|
-
"line":
|
|
10412
|
+
"line": 285
|
|
10396
10413
|
},
|
|
10397
10414
|
"parameters": [
|
|
10398
10415
|
{
|
|
@@ -10421,7 +10438,7 @@
|
|
|
10421
10438
|
"kind": "class",
|
|
10422
10439
|
"locationInModule": {
|
|
10423
10440
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10424
|
-
"line":
|
|
10441
|
+
"line": 259
|
|
10425
10442
|
},
|
|
10426
10443
|
"methods": [
|
|
10427
10444
|
{
|
|
@@ -10432,7 +10449,7 @@
|
|
|
10432
10449
|
},
|
|
10433
10450
|
"locationInModule": {
|
|
10434
10451
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10435
|
-
"line":
|
|
10452
|
+
"line": 358
|
|
10436
10453
|
},
|
|
10437
10454
|
"name": "fromIcebergTableAttributes",
|
|
10438
10455
|
"parameters": [
|
|
@@ -10469,7 +10486,7 @@
|
|
|
10469
10486
|
},
|
|
10470
10487
|
"locationInModule": {
|
|
10471
10488
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10472
|
-
"line":
|
|
10489
|
+
"line": 437
|
|
10473
10490
|
},
|
|
10474
10491
|
"name": "grantRead",
|
|
10475
10492
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -10494,7 +10511,7 @@
|
|
|
10494
10511
|
},
|
|
10495
10512
|
"locationInModule": {
|
|
10496
10513
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10497
|
-
"line":
|
|
10514
|
+
"line": 465
|
|
10498
10515
|
},
|
|
10499
10516
|
"name": "grantReadWrite",
|
|
10500
10517
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -10519,7 +10536,7 @@
|
|
|
10519
10536
|
},
|
|
10520
10537
|
"locationInModule": {
|
|
10521
10538
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10522
|
-
"line":
|
|
10539
|
+
"line": 451
|
|
10523
10540
|
},
|
|
10524
10541
|
"name": "grantWrite",
|
|
10525
10542
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -10548,7 +10565,7 @@
|
|
|
10548
10565
|
"immutable": true,
|
|
10549
10566
|
"locationInModule": {
|
|
10550
10567
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10551
|
-
"line":
|
|
10568
|
+
"line": 267
|
|
10552
10569
|
},
|
|
10553
10570
|
"name": "database",
|
|
10554
10571
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -10564,7 +10581,7 @@
|
|
|
10564
10581
|
"immutable": true,
|
|
10565
10582
|
"locationInModule": {
|
|
10566
10583
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10567
|
-
"line":
|
|
10584
|
+
"line": 273
|
|
10568
10585
|
},
|
|
10569
10586
|
"name": "dataFormat",
|
|
10570
10587
|
"type": {
|
|
@@ -10579,7 +10596,7 @@
|
|
|
10579
10596
|
"immutable": true,
|
|
10580
10597
|
"locationInModule": {
|
|
10581
10598
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10582
|
-
"line":
|
|
10599
|
+
"line": 276
|
|
10583
10600
|
},
|
|
10584
10601
|
"name": "formatVersion",
|
|
10585
10602
|
"type": {
|
|
@@ -10594,7 +10611,7 @@
|
|
|
10594
10611
|
"immutable": true,
|
|
10595
10612
|
"locationInModule": {
|
|
10596
10613
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10597
|
-
"line":
|
|
10614
|
+
"line": 270
|
|
10598
10615
|
},
|
|
10599
10616
|
"name": "location",
|
|
10600
10617
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -10610,7 +10627,7 @@
|
|
|
10610
10627
|
"immutable": true,
|
|
10611
10628
|
"locationInModule": {
|
|
10612
10629
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10613
|
-
"line":
|
|
10630
|
+
"line": 279
|
|
10614
10631
|
},
|
|
10615
10632
|
"name": "resource",
|
|
10616
10633
|
"type": {
|
|
@@ -10625,7 +10642,7 @@
|
|
|
10625
10642
|
"immutable": true,
|
|
10626
10643
|
"locationInModule": {
|
|
10627
10644
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10628
|
-
"line":
|
|
10645
|
+
"line": 261
|
|
10629
10646
|
},
|
|
10630
10647
|
"name": "tableArn",
|
|
10631
10648
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -10641,7 +10658,7 @@
|
|
|
10641
10658
|
"immutable": true,
|
|
10642
10659
|
"locationInModule": {
|
|
10643
10660
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10644
|
-
"line":
|
|
10661
|
+
"line": 264
|
|
10645
10662
|
},
|
|
10646
10663
|
"name": "tableName",
|
|
10647
10664
|
"overrides": "cdk-glue-iceberg-table.IIcebergTable",
|
|
@@ -10663,7 +10680,7 @@
|
|
|
10663
10680
|
"kind": "interface",
|
|
10664
10681
|
"locationInModule": {
|
|
10665
10682
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10666
|
-
"line":
|
|
10683
|
+
"line": 489
|
|
10667
10684
|
},
|
|
10668
10685
|
"name": "IcebergTableAttributes",
|
|
10669
10686
|
"properties": [
|
|
@@ -10676,7 +10693,7 @@
|
|
|
10676
10693
|
"immutable": true,
|
|
10677
10694
|
"locationInModule": {
|
|
10678
10695
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10679
|
-
"line":
|
|
10696
|
+
"line": 491
|
|
10680
10697
|
},
|
|
10681
10698
|
"name": "database",
|
|
10682
10699
|
"type": {
|
|
@@ -10692,7 +10709,7 @@
|
|
|
10692
10709
|
"immutable": true,
|
|
10693
10710
|
"locationInModule": {
|
|
10694
10711
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10695
|
-
"line":
|
|
10712
|
+
"line": 497
|
|
10696
10713
|
},
|
|
10697
10714
|
"name": "location",
|
|
10698
10715
|
"type": {
|
|
@@ -10708,7 +10725,7 @@
|
|
|
10708
10725
|
"immutable": true,
|
|
10709
10726
|
"locationInModule": {
|
|
10710
10727
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10711
|
-
"line":
|
|
10728
|
+
"line": 494
|
|
10712
10729
|
},
|
|
10713
10730
|
"name": "tableName",
|
|
10714
10731
|
"type": {
|
|
@@ -10729,7 +10746,7 @@
|
|
|
10729
10746
|
"kind": "interface",
|
|
10730
10747
|
"locationInModule": {
|
|
10731
10748
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10732
|
-
"line":
|
|
10749
|
+
"line": 161
|
|
10733
10750
|
},
|
|
10734
10751
|
"name": "IcebergTableProps",
|
|
10735
10752
|
"properties": [
|
|
@@ -10743,7 +10760,7 @@
|
|
|
10743
10760
|
"immutable": true,
|
|
10744
10761
|
"locationInModule": {
|
|
10745
10762
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10746
|
-
"line":
|
|
10763
|
+
"line": 169
|
|
10747
10764
|
},
|
|
10748
10765
|
"name": "columns",
|
|
10749
10766
|
"type": {
|
|
@@ -10764,7 +10781,7 @@
|
|
|
10764
10781
|
"immutable": true,
|
|
10765
10782
|
"locationInModule": {
|
|
10766
10783
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10767
|
-
"line":
|
|
10784
|
+
"line": 163
|
|
10768
10785
|
},
|
|
10769
10786
|
"name": "database",
|
|
10770
10787
|
"type": {
|
|
@@ -10781,7 +10798,7 @@
|
|
|
10781
10798
|
"immutable": true,
|
|
10782
10799
|
"locationInModule": {
|
|
10783
10800
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10784
|
-
"line":
|
|
10801
|
+
"line": 191
|
|
10785
10802
|
},
|
|
10786
10803
|
"name": "formatVersion",
|
|
10787
10804
|
"type": {
|
|
@@ -10798,7 +10815,7 @@
|
|
|
10798
10815
|
"immutable": true,
|
|
10799
10816
|
"locationInModule": {
|
|
10800
10817
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10801
|
-
"line":
|
|
10818
|
+
"line": 172
|
|
10802
10819
|
},
|
|
10803
10820
|
"name": "location",
|
|
10804
10821
|
"type": {
|
|
@@ -10814,7 +10831,7 @@
|
|
|
10814
10831
|
"immutable": true,
|
|
10815
10832
|
"locationInModule": {
|
|
10816
10833
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10817
|
-
"line":
|
|
10834
|
+
"line": 166
|
|
10818
10835
|
},
|
|
10819
10836
|
"name": "tableName",
|
|
10820
10837
|
"type": {
|
|
@@ -10831,7 +10848,7 @@
|
|
|
10831
10848
|
"immutable": true,
|
|
10832
10849
|
"locationInModule": {
|
|
10833
10850
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10834
|
-
"line":
|
|
10851
|
+
"line": 211
|
|
10835
10852
|
},
|
|
10836
10853
|
"name": "comment",
|
|
10837
10854
|
"optional": true,
|
|
@@ -10849,7 +10866,7 @@
|
|
|
10849
10866
|
"immutable": true,
|
|
10850
10867
|
"locationInModule": {
|
|
10851
10868
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10852
|
-
"line":
|
|
10869
|
+
"line": 188
|
|
10853
10870
|
},
|
|
10854
10871
|
"name": "dataFormat",
|
|
10855
10872
|
"optional": true,
|
|
@@ -10867,7 +10884,7 @@
|
|
|
10867
10884
|
"immutable": true,
|
|
10868
10885
|
"locationInModule": {
|
|
10869
10886
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10870
|
-
"line":
|
|
10887
|
+
"line": 185
|
|
10871
10888
|
},
|
|
10872
10889
|
"name": "identifierFieldNames",
|
|
10873
10890
|
"optional": true,
|
|
@@ -10890,7 +10907,7 @@
|
|
|
10890
10907
|
"immutable": true,
|
|
10891
10908
|
"locationInModule": {
|
|
10892
10909
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10893
|
-
"line":
|
|
10910
|
+
"line": 175
|
|
10894
10911
|
},
|
|
10895
10912
|
"name": "partitionSpec",
|
|
10896
10913
|
"optional": true,
|
|
@@ -10913,7 +10930,7 @@
|
|
|
10913
10930
|
"immutable": true,
|
|
10914
10931
|
"locationInModule": {
|
|
10915
10932
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10916
|
-
"line":
|
|
10933
|
+
"line": 218
|
|
10917
10934
|
},
|
|
10918
10935
|
"name": "removalPolicy",
|
|
10919
10936
|
"optional": true,
|
|
@@ -10931,7 +10948,7 @@
|
|
|
10931
10948
|
"immutable": true,
|
|
10932
10949
|
"locationInModule": {
|
|
10933
10950
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10934
|
-
"line":
|
|
10951
|
+
"line": 178
|
|
10935
10952
|
},
|
|
10936
10953
|
"name": "sortOrder",
|
|
10937
10954
|
"optional": true,
|
|
@@ -10954,7 +10971,7 @@
|
|
|
10954
10971
|
"immutable": true,
|
|
10955
10972
|
"locationInModule": {
|
|
10956
10973
|
"filename": "lib/iceberg/iceberg-table.ts",
|
|
10957
|
-
"line":
|
|
10974
|
+
"line": 200
|
|
10958
10975
|
},
|
|
10959
10976
|
"name": "tableProperties",
|
|
10960
10977
|
"optional": true,
|
|
@@ -11194,7 +11211,7 @@
|
|
|
11194
11211
|
},
|
|
11195
11212
|
"locationInModule": {
|
|
11196
11213
|
"filename": "lib/iceberg/iceberg-type.ts",
|
|
11197
|
-
"line":
|
|
11214
|
+
"line": 447
|
|
11198
11215
|
},
|
|
11199
11216
|
"name": "isDecimal",
|
|
11200
11217
|
"returns": {
|
|
@@ -11210,7 +11227,7 @@
|
|
|
11210
11227
|
},
|
|
11211
11228
|
"locationInModule": {
|
|
11212
11229
|
"filename": "lib/iceberg/iceberg-type.ts",
|
|
11213
|
-
"line":
|
|
11230
|
+
"line": 452
|
|
11214
11231
|
},
|
|
11215
11232
|
"name": "isFixed",
|
|
11216
11233
|
"returns": {
|
|
@@ -11226,7 +11243,7 @@
|
|
|
11226
11243
|
},
|
|
11227
11244
|
"locationInModule": {
|
|
11228
11245
|
"filename": "lib/iceberg/iceberg-type.ts",
|
|
11229
|
-
"line":
|
|
11246
|
+
"line": 442
|
|
11230
11247
|
},
|
|
11231
11248
|
"name": "isPrimitive",
|
|
11232
11249
|
"parameters": [
|
|
@@ -11937,6 +11954,6 @@
|
|
|
11937
11954
|
"symbolId": "lib/iceberg/iceberg-type:IcebergTypeOptions"
|
|
11938
11955
|
}
|
|
11939
11956
|
},
|
|
11940
|
-
"version": "0.
|
|
11941
|
-
"fingerprint": "
|
|
11957
|
+
"version": "0.6.0",
|
|
11958
|
+
"fingerprint": "/W1xtst59HUo07qLX5mmYrbn1/GdiwicoNOjp3vsDNw="
|
|
11942
11959
|
}
|