cdk-glue-iceberg-table 0.1.7 → 0.1.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -75,13 +75,13 @@ Consumer-facing reference sections below:
75
75
  This repo is **both** the published package and a CDK demo app:
76
76
 
77
77
  - `lib/iceberg/` — the published package (`cdk-glue-iceberg-table` on npm).
78
- - `lib/arceus-stack.ts`, `lib/iceberg-evolution-stack.ts`, `bin/`, `scripts/` — a CDK app that dogfoods the construct against a real AWS account, plus a bash harness that drives schema + partition evolution through real `cdk deploy`s. **Repo-only, not published to npm.**
79
- - `e2e-consumer/` — a standalone CDK app that depends on the **published** `cdk-glue-iceberg-table` from npm. Proves that a fresh install + import + `cdk synth` works for downstream consumers. Runs on every PR via the `e2e-consumer` job in `.github/workflows/ci.yml`. Its `lib/surface-reference.ts` touches every exported symbol so that a rename anywhere in the published surface breaks CI.
78
+ - `lib/arceus-stack.ts`, `lib/iceberg-evolution-stack.ts`, `lib/iceberg-dml-stack.ts`, `bin/`, `scripts/` — a CDK app that dogfoods the construct against a real AWS account, plus two bash harnesses: one drives schema + partition evolution through real `cdk deploy`s, the other exercises the v2 DML surface (UPDATE / DELETE / MERGE / time travel / OPTIMIZE / VACUUM) on a single deploy. **Repo-only, not published to npm.**
79
+ - `e2e-consumer/` — a standalone CDK app that depends on the **published** `cdk-glue-iceberg-table` from npm. Proves that a fresh install + import + `cdk synth` works for downstream consumers. Runs on every PR via the `e2e-consumer` job in `.github/workflows/ci.yml`. Its `lib/surface-reference.ts` touches every exported symbol so that a rename in the published surface breaks CI. The pin in `e2e-consumer/package-lock.json` tracks the version most recently published to npm; CLAUDE.md asks for it to be bumped after each release.
80
80
 
81
81
  How the test gates fit together:
82
82
 
83
- - **`ci.yml`** runs on every PR — lint, unit tests with 95% coverage gate, `npm pack`, and the `e2e-consumer` synth against the latest npm version.
84
- - **`integ-test.yml`** is the real-AWS gate (`scripts/integration-test-evolution.sh` driving four `cdk deploy`s). Gated by `run-integ-test` label or `/run-integ-test` collaborator comment. PRs that touch any file under `lib/`, `bin/arceus.ts`, `cdk.json`, or the script must show a green run before merging (see CLAUDE.md §"Integration test for construct-touching PRs"). Doc-only PRs are exempt.
83
+ - **`ci.yml`** runs on every PR — lint, unit tests with the 95% coverage gate, `npm pack`, and the `e2e-consumer` synth against the pinned published npm version.
84
+ - **`integ-test.yml`** is the real-AWS gate. Runs `scripts/integration-test-evolution.sh` (four `cdk deploy`s exercising schema + partition evolution), then `scripts/integration-test-dml.sh` (one deploy, then UPDATE / DELETE / MERGE / time travel / OPTIMIZE / VACUUM against a v2 merge-on-read table). Gated by `run-integ-test` label or `/run-integ-test` collaborator comment. PRs that touch any file under `lib/`, `bin/arceus.ts`, `cdk.json`, or either script must show a green run before merging (see CLAUDE.md §"Integration test for construct-touching PRs"). Doc-only PRs are exempt.
85
85
  - **`publish.yml`** runs on push to `main` — trusted-publish to npm when `package.json`'s `version` is newer than the registry.
86
86
 
87
87
  The sections [Prerequisites](#prerequisites), [Quickstart](#quickstart),
@@ -100,12 +100,12 @@ Before running the quickstart you need:
100
100
  environment (the AWS CLI sets these automatically for most
101
101
  profile setups; `cdk` also populates them from the active profile).
102
102
  3. `PRINCIPAL_ARN` set to the **ARN of an existing IAM principal**
103
- in this account — IAM user, role, or federated identity. The
103
+ in this account (IAM user, role, or federated identity). The
104
104
  stack adds that principal as a Lake Formation admin and grants
105
105
  it per-table `SELECT/INSERT/DELETE/ALTER/DESCRIBE` on the demo
106
106
  Iceberg tables. Without it the deploy fails when LF can't resolve
107
107
  the principal. The same ARN must also be the identity running
108
- `cdk deploy` and any subsequent Athena queries — local devs
108
+ `cdk deploy` and any subsequent Athena queries. Local devs
109
109
  typically set this to their IAM user ARN; CI (`integ-test.yml`)
110
110
  sets it to the OIDC role ARN.
111
111
  - **Only one principal is privileged at a time.** `ArceusStack`
@@ -131,12 +131,13 @@ Before running the quickstart you need:
131
131
 
132
132
  Repo source paths (npm consumers import from the package root and get the same exports under `dist/lib/iceberg/`):
133
133
 
134
- - **`lib/iceberg/iceberg-table.ts`** — the `IcebergTable` L2 construct.
135
- - **`lib/iceberg/iceberg-type.ts`** — `IcebergType` with primitives + `list` / `map` / `struct` factories. Renders to the JSON shape Glue's `IcebergStructField.type` expects.
136
- - **`lib/iceberg/iceberg-partition-transform.ts`** — `IcebergPartitionTransform` (identity / bucket(N) / truncate(W) / year / month / day / hour / void). Each transform validates against the source column type at synth time.
137
- - **`lib/iceberg/iceberg-table-properties.ts`** — `IcebergDataFormat` (parquet/orc/avro — default parquet), `IcebergFormatVersion` (v1/v2 — default v2), and a validator that catches misconfigured `tableProperties` before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).
138
- - **`lib/arceus-stack.ts`** — the demo stack: KMS-encrypted data lake bucket, Athena results bucket, Glue database, three demo Iceberg tables (`orders`, `events`, `customers`). Repo-only, not in the npm tarball.
139
- - **`lib/iceberg-evolution-stack.ts`** + **`scripts/integration-test-evolution.sh`** — a parameterized stack and a bash harness that drives four real `cdk deploy`s to prove schema/partition evolution works end-to-end. Repo-only, not in the npm tarball.
134
+ - **`lib/iceberg/iceberg-table.ts`**: the `IcebergTable` L2 construct.
135
+ - **`lib/iceberg/iceberg-type.ts`**: `IcebergType` with primitives + `list` / `map` / `struct` factories. Renders to the JSON shape Glue's `IcebergStructField.type` expects.
136
+ - **`lib/iceberg/iceberg-partition-transform.ts`**: `IcebergPartitionTransform` (identity / bucket(N) / truncate(W) / year / month / day / hour / void). Each transform validates against the source column type at synth time.
137
+ - **`lib/iceberg/iceberg-table-properties.ts`**: `IcebergDataFormat` (parquet/orc/avro, default parquet), `IcebergFormatVersion` (v1/v2, default v2), and a validator that catches misconfigured `tableProperties` before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).
138
+ - **`lib/arceus-stack.ts`**: the demo stack: KMS-encrypted data lake bucket, Athena results bucket, Glue database, three demo Iceberg tables (`orders`, `events`, `customers`). Repo-only, not in the npm tarball.
139
+ - **`lib/iceberg-evolution-stack.ts`** + **`scripts/integration-test-evolution.sh`**: a parameterized stack and a bash harness that drives four real `cdk deploy`s to prove schema/partition evolution works end-to-end. Repo-only, not in the npm tarball.
140
+ - **`lib/iceberg-dml-stack.ts`** + **`scripts/integration-test-dml.sh`**: a v2 merge-on-read table and a bash harness that exercises UPDATE, DELETE, MERGE INTO (upsert), time-travel SELECT, OPTIMIZE compaction, and VACUUM snapshot expiration. Single deploy, ~3 minutes. Repo-only, not in the npm tarball.
140
141
 
141
142
  ## Quickstart
142
143
 
@@ -160,16 +161,18 @@ Repo source paths (npm consumers import from the package root and get the same e
160
161
  export PRINCIPAL_ARN="$(aws sts get-caller-identity --query Arn --output text)"
161
162
 
162
163
  npm install
163
- npx jest # 100% line + branch coverage, 95% gate
164
+ npx jest # runs the suite; coverage floor in jest.config.js
164
165
  npx cdk deploy ArceusStack --require-approval=never
165
166
  ./scripts/integration-test-evolution.sh # add + rename + drop, via cdk only
166
167
  ```
167
168
 
168
- `cdk ls` will show two stacks — `ArceusStack` (the demo data lake +
169
- three Iceberg tables) and `IcebergEvolutionStack` (the evolution
170
- test target driven by `scripts/integration-test-evolution.sh`).
171
- Deploy only `ArceusStack` for the quickstart; the evolution stack
172
- is created on demand by the script.
169
+ `cdk ls` will show three stacks: `ArceusStack` (the demo data lake +
170
+ three Iceberg tables), `IcebergEvolutionStack` (the evolution test
171
+ target driven by `scripts/integration-test-evolution.sh`), and
172
+ `IcebergDmlStack` (the DML test target driven by
173
+ `scripts/integration-test-dml.sh`). Deploy only `ArceusStack` for
174
+ the quickstart; the two test stacks are created on demand by their
175
+ respective scripts.
173
176
 
174
177
  ## Using `IcebergTable`
175
178
 
@@ -215,8 +218,8 @@ new IcebergTable(this, 'Users', {
215
218
  });
216
219
  ```
217
220
 
218
- A table that exercises most of the surface — partitions, sort order,
219
- nested types, identifier fields, table properties, removal policy.
221
+ A table that exercises most of the surface (partitions, sort order,
222
+ nested types, identifier fields, table properties, removal policy).
220
223
  This is the exact shape `ArceusStack` uses for the `orders` demo
221
224
  table, so the column list / partition spec / properties round-trip
222
225
  straight to the live metadata.json below.
@@ -488,7 +491,7 @@ underlying Iceberg `metadata.json` after each:
488
491
  | 3 | **RENAME** `email` → `contact_email` (id 2 preserved), **ADD** partition `bucket(8)(customer_id)` | rename | + `bucket(8)(customer_id)` |
489
492
  | 4 | **DROP** column `region` (id 4 stays retired), **DROP** partition `bucket(8)(customer_id)` | − `region` | − `customer_id_bucket` |
490
493
 
491
- Last script run output (abridged — `cdk deploy` chatter and the
494
+ Last script run output (abridged: `cdk deploy` chatter and the
492
495
  per-Athena-query state polling lines are omitted; the assertion
493
496
  output is verbatim):
494
497
 
@@ -539,6 +542,30 @@ metadata delta (new `schema-id`, new `spec-id`) and writes a new
539
542
  the construct pins (`id: N` on each `IcebergColumn`) never change
540
543
  across deploys.
541
544
 
545
+ ## DML, time travel, OPTIMIZE, and VACUUM
546
+
547
+ `scripts/integration-test-dml.sh` covers the v2 surface that the
548
+ evolution test doesn't. The harness deploys `IcebergDmlStack` once,
549
+ then runs a sequence of Athena statements against a v2 merge-on-read
550
+ table with `identifierFieldNames: ['account_id']`:
551
+
552
+ | Step | Statement | Verify |
553
+ | ---: | --- | --- |
554
+ | 1 | `INSERT` 5 seed rows | row count == 5 |
555
+ | 2 | `UPDATE balance WHERE account_id = 2` | balance == 250, row count == 5 |
556
+ | 3 | `DELETE WHERE account_id = 4` | row count == 4, account_id=4 gone |
557
+ | 4 | capture pre-MERGE snapshot id from `dml_test$snapshots` | — |
558
+ | 5 | `MERGE INTO ... USING ... ON account_id` (update id=3, insert id=6 + id=7) | row count == 6, id=3 balance updated, ids 6 and 7 present |
559
+ | 6 | `SELECT ... FOR VERSION AS OF <pre-MERGE snapshot>` | time-travel still sees 4 rows, no id=6 |
560
+ | 7 | `OPTIMIZE ... REWRITE DATA USING BIN_PACK` | succeeds, row count unchanged |
561
+ | 8 | `VACUUM ...` (after a 65-s sleep to clear `max-snapshot-age-ms`) | succeeds, row count unchanged |
562
+ | 9 | final `SELECT` | 6 rows with the expected balances |
563
+
564
+ The DML table is configured with `history.expire.max-snapshot-age-ms`
565
+ of 60 seconds so VACUUM has snapshots to expire on a fresh table.
566
+ Both scripts run sequentially in the same `integ-test.yml` job; the
567
+ evolution test finishes in ~5 minutes and the DML test in ~3.
568
+
542
569
  ## Two footguns the construct prevents
543
570
 
544
571
  ### Footgun #1 — schema under `storageDescriptor.columns`
@@ -583,16 +610,17 @@ under `storageDescriptor`.
583
610
  Even setting just `tableInput: { name: 'foo' }` next to
584
611
  `openTableFormatInput` returns
585
612
  `"Table metadata is expected only via TableInput or via IcebergTableInputProperties inside OpenTableFormatInput"`.
586
- The construct never emits `tableInput` — table-level comment goes
613
+ The construct never emits `tableInput`; the table-level comment goes
587
614
  into `tableProperties['comment']`, which lives inside
588
615
  `icebergTableInput.properties`.
589
616
 
590
- (There is a third footgun — field-id reuse after a column drop — that
617
+ (There is a third footgun, field-id reuse after a column drop, that
591
618
  the construct does **not** prevent. See the next section.)
592
619
 
593
620
  ## Known limitations
594
621
 
595
622
  - **Field-id reuse is not detected across deploys.** If you drop a column with `id = 5` and then add a different column with `id = 5` in a later deploy, Glue accepts the UPDATE and Iceberg's metadata silently violates the "never reuse a retired id" invariant. Readers projecting old snapshots will surface deleted data under the new field's name. The construct enforces uniqueness **within one deploy** (`duplicate column id N` validator), but it doesn't compare against the live table state. The safe workflow is to always pin `id` explicitly and treat dropped ids as retired forever; never let CDK reassign an id that has ever been used.
623
+ - **Partition field ids are positional and not pinnable.** The construct allocates partition `fieldId` densely from 1000 in the order partitions appear in `partitionSpec`. Reordering the array across deploys reassigns those ids for unchanged logical partitions, which is the partition-spec analog of the column-id-reuse footgun above. There is no `IcebergPartitionField.fieldId` pinning prop today. The safe workflow is append-only: add new partition fields at the end of `partitionSpec`, and only drop the trailing ones.
596
624
  - **CREATE-only metadata operation.** The CFN `IcebergInput.metadataOperation` only accepts `CREATE`; the construct always emits that. Subsequent deploys use Glue's normal `UpdateTable` path, which writes new Iceberg metadata in-place.
597
625
  - **Format version is immutable after CREATE.** The `formatVersion` prop is read once at table creation; changing it later requires a destroy + recreate.
598
626
  - **`merge-on-read` requires v2.** The construct rejects `write.{delete,update,merge}.mode = merge-on-read` on a v1 table at synth time.
@@ -608,7 +636,7 @@ npx jest # runs every suite under test/, prints coverage at the end
608
636
  Coverage is gated at 95% statements / 95% branches / 95% lines / 95% functions
609
637
  on `lib/**/*.ts` via the `coverageThreshold.global` block in `jest.config.js`.
610
638
  A failing gate fails the suite, so the README does not paste a transcript that
611
- would drift after the next refactor — run the command locally for the live
639
+ would drift after the next refactor; run the command locally for the live
612
640
  numbers.
613
641
 
614
642
  ## Project layout
@@ -619,6 +647,7 @@ arceus/
619
647
  ├── lib/
620
648
  │ ├── arceus-stack.ts # Demo stack (buckets, DB, 3 demo tables)
621
649
  │ ├── iceberg-evolution-stack.ts # Parameterized stack for the evolution test
650
+ │ ├── iceberg-dml-stack.ts # Stack for the DML / time-travel / OPTIMIZE / VACUUM test
622
651
  │ └── iceberg/
623
652
  │ ├── iceberg-table.ts # The L2 construct itself
624
653
  │ ├── iceberg-type.ts # IcebergType + struct/list/map/decimal/fixed
@@ -628,6 +657,7 @@ arceus/
628
657
  ├── test/
629
658
  │ ├── arceus-stack.test.ts
630
659
  │ ├── iceberg-evolution-stack.test.ts
660
+ │ ├── iceberg-dml-stack.test.ts
631
661
  │ └── iceberg/
632
662
  │ ├── iceberg-partition-transform.test.ts
633
663
  │ ├── iceberg-table-properties.test.ts
@@ -643,7 +673,8 @@ arceus/
643
673
  │ └── lib/surface-reference.ts # Anchors every exported symbol so a
644
674
  │ # rename in the published surface breaks CI
645
675
  ├── scripts/
646
- │ └── integration-test-evolution.sh # End-to-end evolution harness
676
+ │ ├── integration-test-evolution.sh # End-to-end evolution harness
677
+ │ └── integration-test-dml.sh # End-to-end DML harness
647
678
  ├── docs/
648
679
  │ └── integ-test-setup.md # AWS-side prerequisites for the integ-test
649
680
  │ # workflow (OIDC provider, IAM role, repo var)
@@ -657,5 +688,5 @@ arceus/
657
688
  ├── tsconfig.json # Wide include (used by ESLint and dev)
658
689
  ├── tsconfig.build.json # Narrow include (used by `npm publish`)
659
690
  ├── jest.config.js
660
- └── eslint.config.mjs
691
+ └── eslint.config.js
661
692
  ```
@@ -0,0 +1,33 @@
1
+ import { Grant, IGrantable } from 'aws-cdk-lib/aws-iam';
2
+ export declare const READ_TABLE_ACTIONS: string[];
3
+ export declare const WRITE_TABLE_ACTIONS: string[];
4
+ export declare const READ_S3_LIST_ACTIONS: string[];
5
+ export declare const READ_S3_BUCKET_ACTIONS: string[];
6
+ export declare const WRITE_S3_BUCKET_ACTIONS: string[];
7
+ export declare const READ_S3_OBJECT_ACTIONS: string[];
8
+ export declare const WRITE_S3_OBJECT_ACTIONS: string[];
9
+ /**
10
+ * Issue the four policy statements that scope an Iceberg-table grant
11
+ * correctly: Glue actions on the table ARN, S3 list-bucket actions
12
+ * on the bucket ARN with an `s3:prefix` condition so only the
13
+ * table's own prefix can be listed, S3 bucket-level actions that DO
14
+ * NOT support the `s3:prefix` condition (e.g. `GetBucketLocation`,
15
+ * `ListBucketMultipartUploads`) on the bucket ARN with no condition
16
+ * — including them in the conditioned statement would silently deny
17
+ * them at runtime — and S3 object-level actions on the
18
+ * `bucket/prefix*` ARN. Returns the table-actions grant (any of the
19
+ * four is sufficient for the `Grant` API contract; the rest attach
20
+ * as side effects).
21
+ *
22
+ * @internal
23
+ */
24
+ export declare function grantSplit(grantee: IGrantable, args: {
25
+ tableArn: string;
26
+ bucketArn: string;
27
+ objectArn: string;
28
+ prefixGlob: string;
29
+ tableActions: string[];
30
+ listActions: string[];
31
+ bucketActions: string[];
32
+ objectActions: string[];
33
+ }): Grant;
@@ -0,0 +1,112 @@
1
+ "use strict";
2
+ Object.defineProperty(exports, "__esModule", { value: true });
3
+ exports.WRITE_S3_OBJECT_ACTIONS = exports.READ_S3_OBJECT_ACTIONS = exports.WRITE_S3_BUCKET_ACTIONS = exports.READ_S3_BUCKET_ACTIONS = exports.READ_S3_LIST_ACTIONS = exports.WRITE_TABLE_ACTIONS = exports.READ_TABLE_ACTIONS = void 0;
4
+ exports.grantSplit = grantSplit;
5
+ const aws_iam_1 = require("aws-cdk-lib/aws-iam");
6
+ /// /////////////////////////////////////////////////
7
+ /// /////////////////////////////////////////////////
8
+ // Grant action sets
9
+ exports.READ_TABLE_ACTIONS = [
10
+ 'glue:BatchGetPartition',
11
+ 'glue:GetPartition',
12
+ 'glue:GetPartitions',
13
+ 'glue:GetTable',
14
+ 'glue:GetTables',
15
+ 'glue:GetTableVersion',
16
+ 'glue:GetTableVersions',
17
+ ];
18
+ exports.WRITE_TABLE_ACTIONS = [
19
+ 'glue:BatchCreatePartition',
20
+ 'glue:BatchDeletePartition',
21
+ 'glue:CreatePartition',
22
+ 'glue:DeletePartition',
23
+ 'glue:UpdatePartition',
24
+ 'glue:UpdateTable',
25
+ ];
26
+ /// S3 actions that operate at the bucket level AND support the
27
+ /// `s3:prefix` request condition key (per the S3 docs). Granted on
28
+ /// the bucket ARN with a `StringLike s3:prefix = [<prefix>*, <prefix>]`
29
+ /// condition so the grantee can only list the table's own prefix.
30
+ exports.READ_S3_LIST_ACTIONS = [
31
+ 's3:ListBucket',
32
+ ];
33
+ /// S3 actions that operate at the bucket level but DO NOT support
34
+ /// `s3:prefix`. Granted on the bucket ARN with no condition — adding
35
+ /// one would silently deny these actions at runtime even though they
36
+ /// appear in the policy document.
37
+ exports.READ_S3_BUCKET_ACTIONS = [
38
+ 's3:GetBucketLocation',
39
+ ];
40
+ exports.WRITE_S3_BUCKET_ACTIONS = [
41
+ 's3:ListBucketMultipartUploads',
42
+ ];
43
+ /// S3 actions that operate at the object level. Must be granted on
44
+ /// the `bucket/prefix*` ARN.
45
+ exports.READ_S3_OBJECT_ACTIONS = [
46
+ 's3:GetObject',
47
+ ];
48
+ exports.WRITE_S3_OBJECT_ACTIONS = [
49
+ 's3:PutObject',
50
+ 's3:DeleteObject',
51
+ 's3:AbortMultipartUpload',
52
+ 's3:ListMultipartUploadParts',
53
+ ];
54
+ /**
55
+ * Issue the four policy statements that scope an Iceberg-table grant
56
+ * correctly: Glue actions on the table ARN, S3 list-bucket actions
57
+ * on the bucket ARN with an `s3:prefix` condition so only the
58
+ * table's own prefix can be listed, S3 bucket-level actions that DO
59
+ * NOT support the `s3:prefix` condition (e.g. `GetBucketLocation`,
60
+ * `ListBucketMultipartUploads`) on the bucket ARN with no condition
61
+ * — including them in the conditioned statement would silently deny
62
+ * them at runtime — and S3 object-level actions on the
63
+ * `bucket/prefix*` ARN. Returns the table-actions grant (any of the
64
+ * four is sufficient for the `Grant` API contract; the rest attach
65
+ * as side effects).
66
+ *
67
+ * @internal
68
+ */
69
+ function grantSplit(grantee, args) {
70
+ const tableGrant = aws_iam_1.Grant.addToPrincipal({
71
+ grantee,
72
+ actions: args.tableActions,
73
+ resourceArns: [
74
+ args.tableArn,
75
+ ],
76
+ });
77
+ if (args.listActions.length > 0) {
78
+ aws_iam_1.Grant.addToPrincipal({
79
+ grantee,
80
+ actions: args.listActions,
81
+ resourceArns: [
82
+ args.bucketArn,
83
+ ],
84
+ conditions: {
85
+ StringLike: {
86
+ 's3:prefix': [
87
+ args.prefixGlob,
88
+ args.prefixGlob.replace(/\*$/, ''),
89
+ ],
90
+ },
91
+ },
92
+ });
93
+ }
94
+ if (args.bucketActions.length > 0) {
95
+ aws_iam_1.Grant.addToPrincipal({
96
+ grantee,
97
+ actions: args.bucketActions,
98
+ resourceArns: [
99
+ args.bucketArn,
100
+ ],
101
+ });
102
+ }
103
+ aws_iam_1.Grant.addToPrincipal({
104
+ grantee,
105
+ actions: args.objectActions,
106
+ resourceArns: [
107
+ args.objectArn,
108
+ ],
109
+ });
110
+ return tableGrant;
111
+ }
112
+ //# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiaWNlYmVyZy10YWJsZS1ncmFudHMuanMiLCJzb3VyY2VSb290IjoiIiwic291cmNlcyI6WyIuLi8uLi8uLi9saWIvaWNlYmVyZy9pY2ViZXJnLXRhYmxlLWdyYW50cy50cyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOzs7QUE0RUEsZ0NBc0RDO0FBbElELGlEQUc2QjtBQUU3QixxREFBcUQ7QUFDckQscURBQXFEO0FBQ3JELG9CQUFvQjtBQUVQLFFBQUEsa0JBQWtCLEdBQUc7SUFDOUIsd0JBQXdCO0lBQ3hCLG1CQUFtQjtJQUNuQixvQkFBb0I7SUFDcEIsZUFBZTtJQUNmLGdCQUFnQjtJQUNoQixzQkFBc0I7SUFDdEIsdUJBQXVCO0NBQzFCLENBQUM7QUFFVyxRQUFBLG1CQUFtQixHQUFHO0lBQy9CLDJCQUEyQjtJQUMzQiwyQkFBMkI7SUFDM0Isc0JBQXNCO0lBQ3RCLHNCQUFzQjtJQUN0QixzQkFBc0I7SUFDdEIsa0JBQWtCO0NBQ3JCLENBQUM7QUFFRiwrREFBK0Q7QUFDL0QsbUVBQW1FO0FBQ25FLHdFQUF3RTtBQUN4RSxrRUFBa0U7QUFDckQsUUFBQSxvQkFBb0IsR0FBRztJQUNoQyxlQUFlO0NBQ2xCLENBQUM7QUFFRixrRUFBa0U7QUFDbEUscUVBQXFFO0FBQ3JFLHFFQUFxRTtBQUNyRSxrQ0FBa0M7QUFDckIsUUFBQSxzQkFBc0IsR0FBRztJQUNsQyxzQkFBc0I7Q0FDekIsQ0FBQztBQUVXLFFBQUEsdUJBQXVCLEdBQUc7SUFDbkMsK0JBQStCO0NBQ2xDLENBQUM7QUFFRixtRUFBbUU7QUFDbkUsNkJBQTZCO0FBQ2hCLFFBQUEsc0JBQXNCLEdBQUc7SUFDbEMsY0FBYztDQUNqQixDQUFDO0FBRVcsUUFBQSx1QkFBdUIsR0FBRztJQUNuQyxjQUFjO0lBQ2QsaUJBQWlCO0lBQ2pCLHlCQUF5QjtJQUN6Qiw2QkFBNkI7Q0FDaEMsQ0FBQztBQUVGOzs7Ozs7Ozs7Ozs7OztHQWNHO0FBQ0gsU0FBZ0IsVUFBVSxDQUN0QixPQUFtQixFQUNuQixJQVNDO0lBRUQsTUFBTSxVQUFVLEdBQUcsZUFBSyxDQUFDLGNBQWMsQ0FBQztRQUNwQyxPQUFPO1FBQ1AsT0FBTyxFQUFFLElBQUksQ0FBQyxZQUFZO1FBQzFCLFlBQVksRUFBRTtZQUNWLElBQUksQ0FBQyxRQUFRO1NBQ2hCO0tBQ0osQ0FBQyxDQUFDO0lBQ0gsSUFBSSxJQUFJLENBQUMsV0FBVyxDQUFDLE1BQU0sR0FBRyxDQUFDLEVBQUUsQ0FBQztRQUM5QixlQUFLLENBQUMsY0FBYyxDQUFDO1lBQ2pCLE9BQU87WUFDUCxPQUFPLEVBQUUsSUFBSSxDQUFDLFdBQVc7WUFDekIsWUFBWSxFQUFFO2dCQUNWLElBQUksQ0FBQyxTQUFTO2FBQ2pCO1lBQ0QsVUFBVSxFQUFFO2dCQUNSLFVBQVUsRUFBRTtvQkFDUixXQUFXLEVBQUU7d0JBQ1QsSUFBSSxDQUFDLFVBQVU7d0JBQ2YsSUFBSSxDQUFDLFVBQVUsQ0FBQyxPQUFPLENBQUMsS0FBSyxFQUFFLEVBQUUsQ0FBQztxQkFDckM7aUJBQ0o7YUFDSjtTQUNKLENBQUMsQ0FBQztJQUNQLENBQUM7SUFDRCxJQUFJLElBQUksQ0FBQyxhQUFhLENBQUMsTUFBTSxHQUFHLENBQUMsRUFBRSxDQUFDO1FBQ2hDLGVBQUssQ0FBQyxjQUFjLENBQUM7WUFDakIsT0FBTztZQUNQLE9BQU8sRUFBRSxJQUFJLENBQUMsYUFBYTtZQUMzQixZQUFZLEVBQUU7Z0JBQ1YsSUFBSSxDQUFDLFNBQVM7YUFDakI7U0FDSixDQUFDLENBQUM7SUFDUCxDQUFDO0lBQ0QsZUFBSyxDQUFDLGNBQWMsQ0FBQztRQUNqQixPQUFPO1FBQ1AsT0FBTyxFQUFFLElBQUksQ0FBQyxhQUFhO1FBQzNCLFlBQVksRUFBRTtZQUNWLElBQUksQ0FBQyxTQUFTO1NBQ2pCO0tBQ0osQ0FBQyxDQUFDO0lBQ0gsT0FBTyxVQUFVLENBQUM7QUFDdEIsQ0FBQyIsInNvdXJjZXNDb250ZW50IjpbImltcG9ydCB7XG4gICAgR3JhbnQsXG4gICAgSUdyYW50YWJsZSxcbn0gZnJvbSAnYXdzLWNkay1saWIvYXdzLWlhbSc7XG5cbi8vLyAvLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vXG4vLy8gLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vL1xuLy8gR3JhbnQgYWN0aW9uIHNldHNcblxuZXhwb3J0IGNvbnN0IFJFQURfVEFCTEVfQUNUSU9OUyA9IFtcbiAgICAnZ2x1ZTpCYXRjaEdldFBhcnRpdGlvbicsXG4gICAgJ2dsdWU6R2V0UGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpHZXRQYXJ0aXRpb25zJyxcbiAgICAnZ2x1ZTpHZXRUYWJsZScsXG4gICAgJ2dsdWU6R2V0VGFibGVzJyxcbiAgICAnZ2x1ZTpHZXRUYWJsZVZlcnNpb24nLFxuICAgICdnbHVlOkdldFRhYmxlVmVyc2lvbnMnLFxuXTtcblxuZXhwb3J0IGNvbnN0IFdSSVRFX1RBQkxFX0FDVElPTlMgPSBbXG4gICAgJ2dsdWU6QmF0Y2hDcmVhdGVQYXJ0aXRpb24nLFxuICAgICdnbHVlOkJhdGNoRGVsZXRlUGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpDcmVhdGVQYXJ0aXRpb24nLFxuICAgICdnbHVlOkRlbGV0ZVBhcnRpdGlvbicsXG4gICAgJ2dsdWU6VXBkYXRlUGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpVcGRhdGVUYWJsZScsXG5dO1xuXG4vLy8gUzMgYWN0aW9ucyB0aGF0IG9wZXJhdGUgYXQgdGhlIGJ1Y2tldCBsZXZlbCBBTkQgc3VwcG9ydCB0aGVcbi8vLyBgczM6cHJlZml4YCByZXF1ZXN0IGNvbmRpdGlvbiBrZXkgKHBlciB0aGUgUzMgZG9jcykuIEdyYW50ZWQgb25cbi8vLyB0aGUgYnVja2V0IEFSTiB3aXRoIGEgYFN0cmluZ0xpa2UgczM6cHJlZml4ID0gWzxwcmVmaXg+KiwgPHByZWZpeD5dYFxuLy8vIGNvbmRpdGlvbiBzbyB0aGUgZ3JhbnRlZSBjYW4gb25seSBsaXN0IHRoZSB0YWJsZSdzIG93biBwcmVmaXguXG5leHBvcnQgY29uc3QgUkVBRF9TM19MSVNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOkxpc3RCdWNrZXQnLFxuXTtcblxuLy8vIFMzIGFjdGlvbnMgdGhhdCBvcGVyYXRlIGF0IHRoZSBidWNrZXQgbGV2ZWwgYnV0IERPIE5PVCBzdXBwb3J0XG4vLy8gYHMzOnByZWZpeGAuIEdyYW50ZWQgb24gdGhlIGJ1Y2tldCBBUk4gd2l0aCBubyBjb25kaXRpb24g4oCUIGFkZGluZ1xuLy8vIG9uZSB3b3VsZCBzaWxlbnRseSBkZW55IHRoZXNlIGFjdGlvbnMgYXQgcnVudGltZSBldmVuIHRob3VnaCB0aGV5XG4vLy8gYXBwZWFyIGluIHRoZSBwb2xpY3kgZG9jdW1lbnQuXG5leHBvcnQgY29uc3QgUkVBRF9TM19CVUNLRVRfQUNUSU9OUyA9IFtcbiAgICAnczM6R2V0QnVja2V0TG9jYXRpb24nLFxuXTtcblxuZXhwb3J0IGNvbnN0IFdSSVRFX1MzX0JVQ0tFVF9BQ1RJT05TID0gW1xuICAgICdzMzpMaXN0QnVja2V0TXVsdGlwYXJ0VXBsb2FkcycsXG5dO1xuXG4vLy8gUzMgYWN0aW9ucyB0aGF0IG9wZXJhdGUgYXQgdGhlIG9iamVjdCBsZXZlbC4gTXVzdCBiZSBncmFudGVkIG9uXG4vLy8gdGhlIGBidWNrZXQvcHJlZml4KmAgQVJOLlxuZXhwb3J0IGNvbnN0IFJFQURfUzNfT0JKRUNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOkdldE9iamVjdCcsXG5dO1xuXG5leHBvcnQgY29uc3QgV1JJVEVfUzNfT0JKRUNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOlB1dE9iamVjdCcsXG4gICAgJ3MzOkRlbGV0ZU9iamVjdCcsXG4gICAgJ3MzOkFib3J0TXVsdGlwYXJ0VXBsb2FkJyxcbiAgICAnczM6TGlzdE11bHRpcGFydFVwbG9hZFBhcnRzJyxcbl07XG5cbi8qKlxuICogSXNzdWUgdGhlIGZvdXIgcG9saWN5IHN0YXRlbWVudHMgdGhhdCBzY29wZSBhbiBJY2ViZXJnLXRhYmxlIGdyYW50XG4gKiBjb3JyZWN0bHk6IEdsdWUgYWN0aW9ucyBvbiB0aGUgdGFibGUgQVJOLCBTMyBsaXN0LWJ1Y2tldCBhY3Rpb25zXG4gKiBvbiB0aGUgYnVja2V0IEFSTiB3aXRoIGFuIGBzMzpwcmVmaXhgIGNvbmRpdGlvbiBzbyBvbmx5IHRoZVxuICogdGFibGUncyBvd24gcHJlZml4IGNhbiBiZSBsaXN0ZWQsIFMzIGJ1Y2tldC1sZXZlbCBhY3Rpb25zIHRoYXQgRE9cbiAqIE5PVCBzdXBwb3J0IHRoZSBgczM6cHJlZml4YCBjb25kaXRpb24gKGUuZy4gYEdldEJ1Y2tldExvY2F0aW9uYCxcbiAqIGBMaXN0QnVja2V0TXVsdGlwYXJ0VXBsb2Fkc2ApIG9uIHRoZSBidWNrZXQgQVJOIHdpdGggbm8gY29uZGl0aW9uXG4gKiDigJQgaW5jbHVkaW5nIHRoZW0gaW4gdGhlIGNvbmRpdGlvbmVkIHN0YXRlbWVudCB3b3VsZCBzaWxlbnRseSBkZW55XG4gKiB0aGVtIGF0IHJ1bnRpbWUg4oCUIGFuZCBTMyBvYmplY3QtbGV2ZWwgYWN0aW9ucyBvbiB0aGVcbiAqIGBidWNrZXQvcHJlZml4KmAgQVJOLiBSZXR1cm5zIHRoZSB0YWJsZS1hY3Rpb25zIGdyYW50IChhbnkgb2YgdGhlXG4gKiBmb3VyIGlzIHN1ZmZpY2llbnQgZm9yIHRoZSBgR3JhbnRgIEFQSSBjb250cmFjdDsgdGhlIHJlc3QgYXR0YWNoXG4gKiBhcyBzaWRlIGVmZmVjdHMpLlxuICpcbiAqIEBpbnRlcm5hbFxuICovXG5leHBvcnQgZnVuY3Rpb24gZ3JhbnRTcGxpdChcbiAgICBncmFudGVlOiBJR3JhbnRhYmxlLFxuICAgIGFyZ3M6IHtcbiAgICAgICAgdGFibGVBcm46IHN0cmluZztcbiAgICAgICAgYnVja2V0QXJuOiBzdHJpbmc7XG4gICAgICAgIG9iamVjdEFybjogc3RyaW5nO1xuICAgICAgICBwcmVmaXhHbG9iOiBzdHJpbmc7XG4gICAgICAgIHRhYmxlQWN0aW9uczogc3RyaW5nW107XG4gICAgICAgIGxpc3RBY3Rpb25zOiBzdHJpbmdbXTtcbiAgICAgICAgYnVja2V0QWN0aW9uczogc3RyaW5nW107XG4gICAgICAgIG9iamVjdEFjdGlvbnM6IHN0cmluZ1tdO1xuICAgIH0sXG4pOiBHcmFudCB7XG4gICAgY29uc3QgdGFibGVHcmFudCA9IEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgZ3JhbnRlZSxcbiAgICAgICAgYWN0aW9uczogYXJncy50YWJsZUFjdGlvbnMsXG4gICAgICAgIHJlc291cmNlQXJuczogW1xuICAgICAgICAgICAgYXJncy50YWJsZUFybixcbiAgICAgICAgXSxcbiAgICB9KTtcbiAgICBpZiAoYXJncy5saXN0QWN0aW9ucy5sZW5ndGggPiAwKSB7XG4gICAgICAgIEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgICAgIGdyYW50ZWUsXG4gICAgICAgICAgICBhY3Rpb25zOiBhcmdzLmxpc3RBY3Rpb25zLFxuICAgICAgICAgICAgcmVzb3VyY2VBcm5zOiBbXG4gICAgICAgICAgICAgICAgYXJncy5idWNrZXRBcm4sXG4gICAgICAgICAgICBdLFxuICAgICAgICAgICAgY29uZGl0aW9uczoge1xuICAgICAgICAgICAgICAgIFN0cmluZ0xpa2U6IHtcbiAgICAgICAgICAgICAgICAgICAgJ3MzOnByZWZpeCc6IFtcbiAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3MucHJlZml4R2xvYixcbiAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3MucHJlZml4R2xvYi5yZXBsYWNlKC9cXCokLywgJycpLFxuICAgICAgICAgICAgICAgICAgICBdLFxuICAgICAgICAgICAgICAgIH0sXG4gICAgICAgICAgICB9LFxuICAgICAgICB9KTtcbiAgICB9XG4gICAgaWYgKGFyZ3MuYnVja2V0QWN0aW9ucy5sZW5ndGggPiAwKSB7XG4gICAgICAgIEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgICAgIGdyYW50ZWUsXG4gICAgICAgICAgICBhY3Rpb25zOiBhcmdzLmJ1Y2tldEFjdGlvbnMsXG4gICAgICAgICAgICByZXNvdXJjZUFybnM6IFtcbiAgICAgICAgICAgICAgICBhcmdzLmJ1Y2tldEFybixcbiAgICAgICAgICAgIF0sXG4gICAgICAgIH0pO1xuICAgIH1cbiAgICBHcmFudC5hZGRUb1ByaW5jaXBhbCh7XG4gICAgICAgIGdyYW50ZWUsXG4gICAgICAgIGFjdGlvbnM6IGFyZ3Mub2JqZWN0QWN0aW9ucyxcbiAgICAgICAgcmVzb3VyY2VBcm5zOiBbXG4gICAgICAgICAgICBhcmdzLm9iamVjdEFybixcbiAgICAgICAgXSxcbiAgICB9KTtcbiAgICByZXR1cm4gdGFibGVHcmFudDtcbn1cbiJdfQ==
@@ -0,0 +1,41 @@
1
+ import { CfnTable } from 'aws-cdk-lib/aws-glue';
2
+ import { IcebergType } from './iceberg-type';
3
+ import { IcebergDataFormat, IcebergFormatVersion } from './iceberg-table-properties';
4
+ import { IcebergColumn, IcebergPartitionField, IcebergSortField, IcebergTableProps } from './iceberg-table';
5
+ /** @internal */
6
+ export declare function validateProps(props: IcebergTableProps): void;
7
+ /** @internal */
8
+ export declare function normalizeLocation(location: string): string;
9
+ /** @internal */
10
+ export declare function parseS3Uri(uri: string): {
11
+ bucket: string;
12
+ key: string;
13
+ };
14
+ interface RenderedColumn {
15
+ readonly id: number;
16
+ readonly type: IcebergType;
17
+ }
18
+ /** @internal */
19
+ export interface RenderedSchema {
20
+ readonly fields: CfnTable.IcebergStructFieldProperty[];
21
+ readonly columnByName: Map<string, RenderedColumn>;
22
+ }
23
+ /** @internal */
24
+ export declare function renderSchema(columns: IcebergColumn[]): RenderedSchema;
25
+ /** @internal */
26
+ export declare function validatePartitionSpec(partitionSpec: IcebergPartitionField[] | undefined, columnByName: Map<string, RenderedColumn>): void;
27
+ /** @internal */
28
+ export declare function validateSortOrder(sortOrder: IcebergSortField[] | undefined, columnByName: Map<string, RenderedColumn>): void;
29
+ /** @internal */
30
+ export declare function renderPartitionSpec(partitionSpec: IcebergPartitionField[] | undefined, columnByName: Map<string, RenderedColumn>): CfnTable.IcebergPartitionSpecProperty | undefined;
31
+ /** @internal */
32
+ export declare function renderSortOrder(sortOrder: IcebergSortField[] | undefined, columnByName: Map<string, RenderedColumn>): CfnTable.IcebergSortOrderProperty | undefined;
33
+ /** @internal */
34
+ export declare function resolveIdentifierFieldIds(identifierFieldNames: string[] | undefined, columnByName: Map<string, RenderedColumn>): number[] | undefined;
35
+ /** @internal */
36
+ export declare function mergeProperties(dataFormat: IcebergDataFormat, formatVersion: IcebergFormatVersion, user: {
37
+ [key: string]: string;
38
+ } | undefined, comment: string | undefined): {
39
+ [key: string]: string;
40
+ };
41
+ export {};