cdk-glue-iceberg-table 0.1.6 → 0.1.8

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -76,6 +76,13 @@ This repo is **both** the published package and a CDK demo app:
76
76
 
77
77
  - `lib/iceberg/` — the published package (`cdk-glue-iceberg-table` on npm).
78
78
  - `lib/arceus-stack.ts`, `lib/iceberg-evolution-stack.ts`, `bin/`, `scripts/` — a CDK app that dogfoods the construct against a real AWS account, plus a bash harness that drives schema + partition evolution through real `cdk deploy`s. **Repo-only, not published to npm.**
79
+ - `e2e-consumer/` — a standalone CDK app that depends on the **published** `cdk-glue-iceberg-table` from npm. Proves that a fresh install + import + `cdk synth` works for downstream consumers. Runs on every PR via the `e2e-consumer` job in `.github/workflows/ci.yml`. Its `lib/surface-reference.ts` touches every exported symbol so that a rename in the published surface breaks CI. The pin in `e2e-consumer/package-lock.json` tracks the version most recently published to npm; CLAUDE.md asks for it to be bumped after each release.
80
+
81
+ How the test gates fit together:
82
+
83
+ - **`ci.yml`** runs on every PR — lint, unit tests with the 95% coverage gate, `npm pack`, and the `e2e-consumer` synth against the pinned published npm version.
84
+ - **`integ-test.yml`** is the real-AWS gate (`scripts/integration-test-evolution.sh` driving four `cdk deploy`s). Gated by `run-integ-test` label or `/run-integ-test` collaborator comment. PRs that touch any file under `lib/`, `bin/arceus.ts`, `cdk.json`, or the script must show a green run before merging (see CLAUDE.md §"Integration test for construct-touching PRs"). Doc-only PRs are exempt.
85
+ - **`publish.yml`** runs on push to `main` — trusted-publish to npm when `package.json`'s `version` is newer than the registry.
79
86
 
80
87
  The sections [Prerequisites](#prerequisites), [Quickstart](#quickstart),
81
88
  [Demo tables](#demo-tables-deployed-by-arceusstack), and
@@ -92,11 +99,26 @@ Before running the quickstart you need:
92
99
  2. `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` set in the
93
100
  environment (the AWS CLI sets these automatically for most
94
101
  profile setups; `cdk` also populates them from the active profile).
95
- 3. `DEVELOPER_IAM_USER` set to the name of an **existing IAM user**
96
- in this account. The stack adds that user as a Lake Formation
97
- admin and grants it per-table `SELECT/INSERT/DELETE/ALTER/DESCRIBE`
98
- on the demo Iceberg tables — without it the deploy fails when LF
99
- can't resolve the principal.
102
+ 3. `PRINCIPAL_ARN` set to the **ARN of an existing IAM principal**
103
+ in this account (IAM user, role, or federated identity). The
104
+ stack adds that principal as a Lake Formation admin and grants
105
+ it per-table `SELECT/INSERT/DELETE/ALTER/DESCRIBE` on the demo
106
+ Iceberg tables. Without it the deploy fails when LF can't resolve
107
+ the principal. The same ARN must also be the identity running
108
+ `cdk deploy` and any subsequent Athena queries. Local devs
109
+ typically set this to their IAM user ARN; CI (`integ-test.yml`)
110
+ sets it to the OIDC role ARN.
111
+ - **Only one principal is privileged at a time.** `ArceusStack`
112
+ writes the Lake Formation admin list with REPLACE semantics,
113
+ so the most recent `cdk deploy` wins. A local `cdk deploy` with
114
+ your user ARN revokes the OIDC role's LF grants (and breaks the
115
+ next integ-test run until CI redeploys), and vice versa.
116
+ Coordinate accordingly.
117
+ - **SSO / `aws-vault` users:** set `PRINCIPAL_ARN` to the
118
+ canonical role ARN (`arn:aws:iam::<acct>:role/<RoleName>`), not
119
+ the per-session `arn:aws:sts::<acct>:assumed-role/...` you'd
120
+ get from `aws sts get-caller-identity`. LF grants on the
121
+ session-suffixed ARN go stale at the next SSO refresh.
100
122
  4. The Lake Formation service-linked role
101
123
  `AWSServiceRoleForLakeFormationDataAccess` must exist in the
102
124
  account. Create it once with
@@ -109,31 +131,41 @@ Before running the quickstart you need:
109
131
 
110
132
  Repo source paths (npm consumers import from the package root and get the same exports under `dist/lib/iceberg/`):
111
133
 
112
- - **`lib/iceberg/iceberg-table.ts`** — the `IcebergTable` L2 construct.
113
- - **`lib/iceberg/iceberg-type.ts`** — `IcebergType` with primitives + `list` / `map` / `struct` factories. Renders to the JSON shape Glue's `IcebergStructField.type` expects.
114
- - **`lib/iceberg/iceberg-partition-transform.ts`** — `IcebergPartitionTransform` (identity / bucket(N) / truncate(W) / year / month / day / hour / void). Each transform validates against the source column type at synth time.
115
- - **`lib/iceberg/iceberg-table-properties.ts`** — `IcebergDataFormat` (parquet/orc/avro — default parquet), `IcebergFormatVersion` (v1/v2 — default v2), and a validator that catches misconfigured `tableProperties` before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).
116
- - **`lib/arceus-stack.ts`** — the demo stack: KMS-encrypted data lake bucket, Athena results bucket, Glue database, three demo Iceberg tables (`orders`, `events`, `customers`). Repo-only, not in the npm tarball.
117
- - **`lib/iceberg-evolution-stack.ts`** + **`scripts/integration-test-evolution.sh`** — a parameterized stack and a bash harness that drives four real `cdk deploy`s to prove schema/partition evolution works end-to-end. Repo-only, not in the npm tarball.
134
+ - **`lib/iceberg/iceberg-table.ts`**: the `IcebergTable` L2 construct.
135
+ - **`lib/iceberg/iceberg-type.ts`**: `IcebergType` with primitives + `list` / `map` / `struct` factories. Renders to the JSON shape Glue's `IcebergStructField.type` expects.
136
+ - **`lib/iceberg/iceberg-partition-transform.ts`**: `IcebergPartitionTransform` (identity / bucket(N) / truncate(W) / year / month / day / hour / void). Each transform validates against the source column type at synth time.
137
+ - **`lib/iceberg/iceberg-table-properties.ts`**: `IcebergDataFormat` (parquet/orc/avro, default parquet), `IcebergFormatVersion` (v1/v2, default v2), and a validator that catches misconfigured `tableProperties` before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).
138
+ - **`lib/arceus-stack.ts`**: the demo stack: KMS-encrypted data lake bucket, Athena results bucket, Glue database, three demo Iceberg tables (`orders`, `events`, `customers`). Repo-only, not in the npm tarball.
139
+ - **`lib/iceberg-evolution-stack.ts`** + **`scripts/integration-test-evolution.sh`**: a parameterized stack and a bash harness that drives four real `cdk deploy`s to prove schema/partition evolution works end-to-end. Repo-only, not in the npm tarball.
118
140
 
119
141
  ## Quickstart
120
142
 
121
143
  ```bash
122
- # DEVELOPER_IAM_USER must be the name of an existing IAM user in
123
- # this account. `aws iam get-user` works when you're authenticated
124
- # directly as an IAM user; for SSO / assumed-role / aws-vault setups
125
- # (the modern default) the call fails with "Must specify userName
126
- # when calling with non-User credentials" and you should set the
127
- # variable manually to whichever IAM user you want LF to grant on.
128
- export DEVELOPER_IAM_USER="$(aws iam get-user --query 'User.UserName' --output text)"
144
+ # PRINCIPAL_ARN is the ARN of the IAM principal (user, role, or
145
+ # federated identity) that the stack should make a Lake Formation
146
+ # admin and per-table grantee. It must equal the identity running
147
+ # `cdk deploy` and any subsequent Athena queries — otherwise the
148
+ # integration script's INSERT/SELECT calls fail with `Principal does
149
+ # not have any privilege on specified resource`.
150
+ #
151
+ # For a direct IAM-user session, the line below returns the right ARN
152
+ # (`arn:aws:iam::<acct>:user/<name>`).
153
+ #
154
+ # For SSO / aws-vault / any assumed-role session this returns
155
+ # `arn:aws:sts::<acct>:assumed-role/<RoleName>/<SessionName>`, which
156
+ # Lake Formation accepts but stales on the next session refresh
157
+ # because the SessionName changes. Set PRINCIPAL_ARN to the canonical
158
+ # `arn:aws:iam::<acct>:role/<RoleName>` instead — for example:
159
+ # export PRINCIPAL_ARN="arn:aws:iam::123456789012:role/MyDevRole"
160
+ export PRINCIPAL_ARN="$(aws sts get-caller-identity --query Arn --output text)"
129
161
 
130
162
  npm install
131
- npx jest # 100% line + branch coverage, 95% gate
163
+ npx jest # runs the suite; coverage floor in jest.config.js
132
164
  npx cdk deploy ArceusStack --require-approval=never
133
165
  ./scripts/integration-test-evolution.sh # add + rename + drop, via cdk only
134
166
  ```
135
167
 
136
- `cdk ls` will show two stacks — `ArceusStack` (the demo data lake +
168
+ `cdk ls` will show two stacks: `ArceusStack` (the demo data lake +
137
169
  three Iceberg tables) and `IcebergEvolutionStack` (the evolution
138
170
  test target driven by `scripts/integration-test-evolution.sh`).
139
171
  Deploy only `ArceusStack` for the quickstart; the evolution stack
@@ -150,7 +182,7 @@ import {
150
182
  import {
151
183
  IcebergTable,
152
184
  IcebergType,
153
- } from './iceberg';
185
+ } from 'cdk-glue-iceberg-table';
154
186
 
155
187
  const db = new Database(this, 'Db', {
156
188
  databaseName: 'analytics',
@@ -183,8 +215,8 @@ new IcebergTable(this, 'Users', {
183
215
  });
184
216
  ```
185
217
 
186
- A table that exercises most of the surface — partitions, sort order,
187
- nested types, identifier fields, table properties, removal policy.
218
+ A table that exercises most of the surface (partitions, sort order,
219
+ nested types, identifier fields, table properties, removal policy).
188
220
  This is the exact shape `ArceusStack` uses for the `orders` demo
189
221
  table, so the column list / partition spec / properties round-trip
190
222
  straight to the live metadata.json below.
@@ -204,7 +236,7 @@ import {
204
236
  IcebergSortDirection,
205
237
  IcebergTable,
206
238
  IcebergType,
207
- } from './iceberg';
239
+ } from 'cdk-glue-iceberg-table';
208
240
 
209
241
  new IcebergTable(this, 'OrdersTable', {
210
242
  database: db,
@@ -456,7 +488,7 @@ underlying Iceberg `metadata.json` after each:
456
488
  | 3 | **RENAME** `email` → `contact_email` (id 2 preserved), **ADD** partition `bucket(8)(customer_id)` | rename | + `bucket(8)(customer_id)` |
457
489
  | 4 | **DROP** column `region` (id 4 stays retired), **DROP** partition `bucket(8)(customer_id)` | − `region` | − `customer_id_bucket` |
458
490
 
459
- Last script run output (abridged — `cdk deploy` chatter and the
491
+ Last script run output (abridged: `cdk deploy` chatter and the
460
492
  per-Athena-query state polling lines are omitted; the assertion
461
493
  output is verbatim):
462
494
 
@@ -551,16 +583,17 @@ under `storageDescriptor`.
551
583
  Even setting just `tableInput: { name: 'foo' }` next to
552
584
  `openTableFormatInput` returns
553
585
  `"Table metadata is expected only via TableInput or via IcebergTableInputProperties inside OpenTableFormatInput"`.
554
- The construct never emits `tableInput` — table-level comment goes
586
+ The construct never emits `tableInput`; the table-level comment goes
555
587
  into `tableProperties['comment']`, which lives inside
556
588
  `icebergTableInput.properties`.
557
589
 
558
- (There is a third footgun — field-id reuse after a column drop — that
590
+ (There is a third footgun, field-id reuse after a column drop, that
559
591
  the construct does **not** prevent. See the next section.)
560
592
 
561
593
  ## Known limitations
562
594
 
563
595
  - **Field-id reuse is not detected across deploys.** If you drop a column with `id = 5` and then add a different column with `id = 5` in a later deploy, Glue accepts the UPDATE and Iceberg's metadata silently violates the "never reuse a retired id" invariant. Readers projecting old snapshots will surface deleted data under the new field's name. The construct enforces uniqueness **within one deploy** (`duplicate column id N` validator), but it doesn't compare against the live table state. The safe workflow is to always pin `id` explicitly and treat dropped ids as retired forever; never let CDK reassign an id that has ever been used.
596
+ - **Partition field ids are positional and not pinnable.** The construct allocates partition `fieldId` densely from 1000 in the order partitions appear in `partitionSpec`. Reordering the array across deploys reassigns those ids for unchanged logical partitions, which is the partition-spec analog of the column-id-reuse footgun above. There is no `IcebergPartitionField.fieldId` pinning prop today. The safe workflow is append-only: add new partition fields at the end of `partitionSpec`, and only drop the trailing ones.
564
597
  - **CREATE-only metadata operation.** The CFN `IcebergInput.metadataOperation` only accepts `CREATE`; the construct always emits that. Subsequent deploys use Glue's normal `UpdateTable` path, which writes new Iceberg metadata in-place.
565
598
  - **Format version is immutable after CREATE.** The `formatVersion` prop is read once at table creation; changing it later requires a destroy + recreate.
566
599
  - **`merge-on-read` requires v2.** The construct rejects `write.{delete,update,merge}.mode = merge-on-read` on a v1 table at synth time.
@@ -576,7 +609,7 @@ npx jest # runs every suite under test/, prints coverage at the end
576
609
  Coverage is gated at 95% statements / 95% branches / 95% lines / 95% functions
577
610
  on `lib/**/*.ts` via the `coverageThreshold.global` block in `jest.config.js`.
578
611
  A failing gate fails the suite, so the README does not paste a transcript that
579
- would drift after the next refactor — run the command locally for the live
612
+ would drift after the next refactor; run the command locally for the live
580
613
  numbers.
581
614
 
582
615
  ## Project layout
@@ -592,7 +625,7 @@ arceus/
592
625
  │ ├── iceberg-type.ts # IcebergType + struct/list/map/decimal/fixed
593
626
  │ ├── iceberg-partition-transform.ts
594
627
  │ ├── iceberg-table-properties.ts # Format/version enums + property validation
595
- │ └── index.ts # Re-exports
628
+ │ └── index.ts # Re-exports (the npm package's entry point)
596
629
  ├── test/
597
630
  │ ├── arceus-stack.test.ts
598
631
  │ ├── iceberg-evolution-stack.test.ts
@@ -601,11 +634,29 @@ arceus/
601
634
  │ ├── iceberg-table-properties.test.ts
602
635
  │ ├── iceberg-table.test.ts
603
636
  │ └── iceberg-type.test.ts
637
+ ├── e2e-consumer/ # Standalone CDK app that consumes the
638
+ │ │ # published npm package — proves a fresh
639
+ │ │ # install + import + synth works. Runs
640
+ │ │ # on every PR via the `e2e-consumer` job
641
+ │ │ # in `.github/workflows/ci.yml`.
642
+ │ ├── bin/app.ts
643
+ │ ├── lib/consumer-stack.ts # Realistic consumer (one IcebergTable)
644
+ │ └── lib/surface-reference.ts # Anchors every exported symbol so a
645
+ │ # rename in the published surface breaks CI
604
646
  ├── scripts/
605
647
  │ └── integration-test-evolution.sh # End-to-end evolution harness
648
+ ├── docs/
649
+ │ └── integ-test-setup.md # AWS-side prerequisites for the integ-test
650
+ │ # workflow (OIDC provider, IAM role, repo var)
651
+ ├── .github/workflows/
652
+ │ ├── ci.yml # Lint + test + build + pack + e2e-consumer
653
+ │ ├── publish.yml # Trusted-publish to npm on version bump
654
+ │ └── integ-test.yml # Real-AWS evolution test (gated by label /
655
+ │ # `/run-integ-test` comment)
606
656
  ├── cdk.json
607
- ├── package.json
657
+ ├── package.json # The published package's manifest
658
+ ├── tsconfig.json # Wide include (used by ESLint and dev)
659
+ ├── tsconfig.build.json # Narrow include (used by `npm publish`)
608
660
  ├── jest.config.js
609
- ├── eslint.config.js
610
- └── tsconfig.json
661
+ └── eslint.config.js
611
662
  ```
@@ -0,0 +1,33 @@
1
+ import { Grant, IGrantable } from 'aws-cdk-lib/aws-iam';
2
+ export declare const READ_TABLE_ACTIONS: string[];
3
+ export declare const WRITE_TABLE_ACTIONS: string[];
4
+ export declare const READ_S3_LIST_ACTIONS: string[];
5
+ export declare const READ_S3_BUCKET_ACTIONS: string[];
6
+ export declare const WRITE_S3_BUCKET_ACTIONS: string[];
7
+ export declare const READ_S3_OBJECT_ACTIONS: string[];
8
+ export declare const WRITE_S3_OBJECT_ACTIONS: string[];
9
+ /**
10
+ * Issue the four policy statements that scope an Iceberg-table grant
11
+ * correctly: Glue actions on the table ARN, S3 list-bucket actions
12
+ * on the bucket ARN with an `s3:prefix` condition so only the
13
+ * table's own prefix can be listed, S3 bucket-level actions that DO
14
+ * NOT support the `s3:prefix` condition (e.g. `GetBucketLocation`,
15
+ * `ListBucketMultipartUploads`) on the bucket ARN with no condition
16
+ * — including them in the conditioned statement would silently deny
17
+ * them at runtime — and S3 object-level actions on the
18
+ * `bucket/prefix*` ARN. Returns the table-actions grant (any of the
19
+ * four is sufficient for the `Grant` API contract; the rest attach
20
+ * as side effects).
21
+ *
22
+ * @internal
23
+ */
24
+ export declare function grantSplit(grantee: IGrantable, args: {
25
+ tableArn: string;
26
+ bucketArn: string;
27
+ objectArn: string;
28
+ prefixGlob: string;
29
+ tableActions: string[];
30
+ listActions: string[];
31
+ bucketActions: string[];
32
+ objectActions: string[];
33
+ }): Grant;
@@ -0,0 +1,112 @@
1
+ "use strict";
2
+ Object.defineProperty(exports, "__esModule", { value: true });
3
+ exports.WRITE_S3_OBJECT_ACTIONS = exports.READ_S3_OBJECT_ACTIONS = exports.WRITE_S3_BUCKET_ACTIONS = exports.READ_S3_BUCKET_ACTIONS = exports.READ_S3_LIST_ACTIONS = exports.WRITE_TABLE_ACTIONS = exports.READ_TABLE_ACTIONS = void 0;
4
+ exports.grantSplit = grantSplit;
5
+ const aws_iam_1 = require("aws-cdk-lib/aws-iam");
6
+ /// /////////////////////////////////////////////////
7
+ /// /////////////////////////////////////////////////
8
+ // Grant action sets
9
+ exports.READ_TABLE_ACTIONS = [
10
+ 'glue:BatchGetPartition',
11
+ 'glue:GetPartition',
12
+ 'glue:GetPartitions',
13
+ 'glue:GetTable',
14
+ 'glue:GetTables',
15
+ 'glue:GetTableVersion',
16
+ 'glue:GetTableVersions',
17
+ ];
18
+ exports.WRITE_TABLE_ACTIONS = [
19
+ 'glue:BatchCreatePartition',
20
+ 'glue:BatchDeletePartition',
21
+ 'glue:CreatePartition',
22
+ 'glue:DeletePartition',
23
+ 'glue:UpdatePartition',
24
+ 'glue:UpdateTable',
25
+ ];
26
+ /// S3 actions that operate at the bucket level AND support the
27
+ /// `s3:prefix` request condition key (per the S3 docs). Granted on
28
+ /// the bucket ARN with a `StringLike s3:prefix = [<prefix>*, <prefix>]`
29
+ /// condition so the grantee can only list the table's own prefix.
30
+ exports.READ_S3_LIST_ACTIONS = [
31
+ 's3:ListBucket',
32
+ ];
33
+ /// S3 actions that operate at the bucket level but DO NOT support
34
+ /// `s3:prefix`. Granted on the bucket ARN with no condition — adding
35
+ /// one would silently deny these actions at runtime even though they
36
+ /// appear in the policy document.
37
+ exports.READ_S3_BUCKET_ACTIONS = [
38
+ 's3:GetBucketLocation',
39
+ ];
40
+ exports.WRITE_S3_BUCKET_ACTIONS = [
41
+ 's3:ListBucketMultipartUploads',
42
+ ];
43
+ /// S3 actions that operate at the object level. Must be granted on
44
+ /// the `bucket/prefix*` ARN.
45
+ exports.READ_S3_OBJECT_ACTIONS = [
46
+ 's3:GetObject',
47
+ ];
48
+ exports.WRITE_S3_OBJECT_ACTIONS = [
49
+ 's3:PutObject',
50
+ 's3:DeleteObject',
51
+ 's3:AbortMultipartUpload',
52
+ 's3:ListMultipartUploadParts',
53
+ ];
54
+ /**
55
+ * Issue the four policy statements that scope an Iceberg-table grant
56
+ * correctly: Glue actions on the table ARN, S3 list-bucket actions
57
+ * on the bucket ARN with an `s3:prefix` condition so only the
58
+ * table's own prefix can be listed, S3 bucket-level actions that DO
59
+ * NOT support the `s3:prefix` condition (e.g. `GetBucketLocation`,
60
+ * `ListBucketMultipartUploads`) on the bucket ARN with no condition
61
+ * — including them in the conditioned statement would silently deny
62
+ * them at runtime — and S3 object-level actions on the
63
+ * `bucket/prefix*` ARN. Returns the table-actions grant (any of the
64
+ * four is sufficient for the `Grant` API contract; the rest attach
65
+ * as side effects).
66
+ *
67
+ * @internal
68
+ */
69
+ function grantSplit(grantee, args) {
70
+ const tableGrant = aws_iam_1.Grant.addToPrincipal({
71
+ grantee,
72
+ actions: args.tableActions,
73
+ resourceArns: [
74
+ args.tableArn,
75
+ ],
76
+ });
77
+ if (args.listActions.length > 0) {
78
+ aws_iam_1.Grant.addToPrincipal({
79
+ grantee,
80
+ actions: args.listActions,
81
+ resourceArns: [
82
+ args.bucketArn,
83
+ ],
84
+ conditions: {
85
+ StringLike: {
86
+ 's3:prefix': [
87
+ args.prefixGlob,
88
+ args.prefixGlob.replace(/\*$/, ''),
89
+ ],
90
+ },
91
+ },
92
+ });
93
+ }
94
+ if (args.bucketActions.length > 0) {
95
+ aws_iam_1.Grant.addToPrincipal({
96
+ grantee,
97
+ actions: args.bucketActions,
98
+ resourceArns: [
99
+ args.bucketArn,
100
+ ],
101
+ });
102
+ }
103
+ aws_iam_1.Grant.addToPrincipal({
104
+ grantee,
105
+ actions: args.objectActions,
106
+ resourceArns: [
107
+ args.objectArn,
108
+ ],
109
+ });
110
+ return tableGrant;
111
+ }
112
+ //# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiaWNlYmVyZy10YWJsZS1ncmFudHMuanMiLCJzb3VyY2VSb290IjoiIiwic291cmNlcyI6WyIuLi8uLi8uLi9saWIvaWNlYmVyZy9pY2ViZXJnLXRhYmxlLWdyYW50cy50cyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOzs7QUE0RUEsZ0NBc0RDO0FBbElELGlEQUc2QjtBQUU3QixxREFBcUQ7QUFDckQscURBQXFEO0FBQ3JELG9CQUFvQjtBQUVQLFFBQUEsa0JBQWtCLEdBQUc7SUFDOUIsd0JBQXdCO0lBQ3hCLG1CQUFtQjtJQUNuQixvQkFBb0I7SUFDcEIsZUFBZTtJQUNmLGdCQUFnQjtJQUNoQixzQkFBc0I7SUFDdEIsdUJBQXVCO0NBQzFCLENBQUM7QUFFVyxRQUFBLG1CQUFtQixHQUFHO0lBQy9CLDJCQUEyQjtJQUMzQiwyQkFBMkI7SUFDM0Isc0JBQXNCO0lBQ3RCLHNCQUFzQjtJQUN0QixzQkFBc0I7SUFDdEIsa0JBQWtCO0NBQ3JCLENBQUM7QUFFRiwrREFBK0Q7QUFDL0QsbUVBQW1FO0FBQ25FLHdFQUF3RTtBQUN4RSxrRUFBa0U7QUFDckQsUUFBQSxvQkFBb0IsR0FBRztJQUNoQyxlQUFlO0NBQ2xCLENBQUM7QUFFRixrRUFBa0U7QUFDbEUscUVBQXFFO0FBQ3JFLHFFQUFxRTtBQUNyRSxrQ0FBa0M7QUFDckIsUUFBQSxzQkFBc0IsR0FBRztJQUNsQyxzQkFBc0I7Q0FDekIsQ0FBQztBQUVXLFFBQUEsdUJBQXVCLEdBQUc7SUFDbkMsK0JBQStCO0NBQ2xDLENBQUM7QUFFRixtRUFBbUU7QUFDbkUsNkJBQTZCO0FBQ2hCLFFBQUEsc0JBQXNCLEdBQUc7SUFDbEMsY0FBYztDQUNqQixDQUFDO0FBRVcsUUFBQSx1QkFBdUIsR0FBRztJQUNuQyxjQUFjO0lBQ2QsaUJBQWlCO0lBQ2pCLHlCQUF5QjtJQUN6Qiw2QkFBNkI7Q0FDaEMsQ0FBQztBQUVGOzs7Ozs7Ozs7Ozs7OztHQWNHO0FBQ0gsU0FBZ0IsVUFBVSxDQUN0QixPQUFtQixFQUNuQixJQVNDO0lBRUQsTUFBTSxVQUFVLEdBQUcsZUFBSyxDQUFDLGNBQWMsQ0FBQztRQUNwQyxPQUFPO1FBQ1AsT0FBTyxFQUFFLElBQUksQ0FBQyxZQUFZO1FBQzFCLFlBQVksRUFBRTtZQUNWLElBQUksQ0FBQyxRQUFRO1NBQ2hCO0tBQ0osQ0FBQyxDQUFDO0lBQ0gsSUFBSSxJQUFJLENBQUMsV0FBVyxDQUFDLE1BQU0sR0FBRyxDQUFDLEVBQUUsQ0FBQztRQUM5QixlQUFLLENBQUMsY0FBYyxDQUFDO1lBQ2pCLE9BQU87WUFDUCxPQUFPLEVBQUUsSUFBSSxDQUFDLFdBQVc7WUFDekIsWUFBWSxFQUFFO2dCQUNWLElBQUksQ0FBQyxTQUFTO2FBQ2pCO1lBQ0QsVUFBVSxFQUFFO2dCQUNSLFVBQVUsRUFBRTtvQkFDUixXQUFXLEVBQUU7d0JBQ1QsSUFBSSxDQUFDLFVBQVU7d0JBQ2YsSUFBSSxDQUFDLFVBQVUsQ0FBQyxPQUFPLENBQUMsS0FBSyxFQUFFLEVBQUUsQ0FBQztxQkFDckM7aUJBQ0o7YUFDSjtTQUNKLENBQUMsQ0FBQztJQUNQLENBQUM7SUFDRCxJQUFJLElBQUksQ0FBQyxhQUFhLENBQUMsTUFBTSxHQUFHLENBQUMsRUFBRSxDQUFDO1FBQ2hDLGVBQUssQ0FBQyxjQUFjLENBQUM7WUFDakIsT0FBTztZQUNQLE9BQU8sRUFBRSxJQUFJLENBQUMsYUFBYTtZQUMzQixZQUFZLEVBQUU7Z0JBQ1YsSUFBSSxDQUFDLFNBQVM7YUFDakI7U0FDSixDQUFDLENBQUM7SUFDUCxDQUFDO0lBQ0QsZUFBSyxDQUFDLGNBQWMsQ0FBQztRQUNqQixPQUFPO1FBQ1AsT0FBTyxFQUFFLElBQUksQ0FBQyxhQUFhO1FBQzNCLFlBQVksRUFBRTtZQUNWLElBQUksQ0FBQyxTQUFTO1NBQ2pCO0tBQ0osQ0FBQyxDQUFDO0lBQ0gsT0FBTyxVQUFVLENBQUM7QUFDdEIsQ0FBQyIsInNvdXJjZXNDb250ZW50IjpbImltcG9ydCB7XG4gICAgR3JhbnQsXG4gICAgSUdyYW50YWJsZSxcbn0gZnJvbSAnYXdzLWNkay1saWIvYXdzLWlhbSc7XG5cbi8vLyAvLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vXG4vLy8gLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vL1xuLy8gR3JhbnQgYWN0aW9uIHNldHNcblxuZXhwb3J0IGNvbnN0IFJFQURfVEFCTEVfQUNUSU9OUyA9IFtcbiAgICAnZ2x1ZTpCYXRjaEdldFBhcnRpdGlvbicsXG4gICAgJ2dsdWU6R2V0UGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpHZXRQYXJ0aXRpb25zJyxcbiAgICAnZ2x1ZTpHZXRUYWJsZScsXG4gICAgJ2dsdWU6R2V0VGFibGVzJyxcbiAgICAnZ2x1ZTpHZXRUYWJsZVZlcnNpb24nLFxuICAgICdnbHVlOkdldFRhYmxlVmVyc2lvbnMnLFxuXTtcblxuZXhwb3J0IGNvbnN0IFdSSVRFX1RBQkxFX0FDVElPTlMgPSBbXG4gICAgJ2dsdWU6QmF0Y2hDcmVhdGVQYXJ0aXRpb24nLFxuICAgICdnbHVlOkJhdGNoRGVsZXRlUGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpDcmVhdGVQYXJ0aXRpb24nLFxuICAgICdnbHVlOkRlbGV0ZVBhcnRpdGlvbicsXG4gICAgJ2dsdWU6VXBkYXRlUGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpVcGRhdGVUYWJsZScsXG5dO1xuXG4vLy8gUzMgYWN0aW9ucyB0aGF0IG9wZXJhdGUgYXQgdGhlIGJ1Y2tldCBsZXZlbCBBTkQgc3VwcG9ydCB0aGVcbi8vLyBgczM6cHJlZml4YCByZXF1ZXN0IGNvbmRpdGlvbiBrZXkgKHBlciB0aGUgUzMgZG9jcykuIEdyYW50ZWQgb25cbi8vLyB0aGUgYnVja2V0IEFSTiB3aXRoIGEgYFN0cmluZ0xpa2UgczM6cHJlZml4ID0gWzxwcmVmaXg+KiwgPHByZWZpeD5dYFxuLy8vIGNvbmRpdGlvbiBzbyB0aGUgZ3JhbnRlZSBjYW4gb25seSBsaXN0IHRoZSB0YWJsZSdzIG93biBwcmVmaXguXG5leHBvcnQgY29uc3QgUkVBRF9TM19MSVNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOkxpc3RCdWNrZXQnLFxuXTtcblxuLy8vIFMzIGFjdGlvbnMgdGhhdCBvcGVyYXRlIGF0IHRoZSBidWNrZXQgbGV2ZWwgYnV0IERPIE5PVCBzdXBwb3J0XG4vLy8gYHMzOnByZWZpeGAuIEdyYW50ZWQgb24gdGhlIGJ1Y2tldCBBUk4gd2l0aCBubyBjb25kaXRpb24g4oCUIGFkZGluZ1xuLy8vIG9uZSB3b3VsZCBzaWxlbnRseSBkZW55IHRoZXNlIGFjdGlvbnMgYXQgcnVudGltZSBldmVuIHRob3VnaCB0aGV5XG4vLy8gYXBwZWFyIGluIHRoZSBwb2xpY3kgZG9jdW1lbnQuXG5leHBvcnQgY29uc3QgUkVBRF9TM19CVUNLRVRfQUNUSU9OUyA9IFtcbiAgICAnczM6R2V0QnVja2V0TG9jYXRpb24nLFxuXTtcblxuZXhwb3J0IGNvbnN0IFdSSVRFX1MzX0JVQ0tFVF9BQ1RJT05TID0gW1xuICAgICdzMzpMaXN0QnVja2V0TXVsdGlwYXJ0VXBsb2FkcycsXG5dO1xuXG4vLy8gUzMgYWN0aW9ucyB0aGF0IG9wZXJhdGUgYXQgdGhlIG9iamVjdCBsZXZlbC4gTXVzdCBiZSBncmFudGVkIG9uXG4vLy8gdGhlIGBidWNrZXQvcHJlZml4KmAgQVJOLlxuZXhwb3J0IGNvbnN0IFJFQURfUzNfT0JKRUNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOkdldE9iamVjdCcsXG5dO1xuXG5leHBvcnQgY29uc3QgV1JJVEVfUzNfT0JKRUNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOlB1dE9iamVjdCcsXG4gICAgJ3MzOkRlbGV0ZU9iamVjdCcsXG4gICAgJ3MzOkFib3J0TXVsdGlwYXJ0VXBsb2FkJyxcbiAgICAnczM6TGlzdE11bHRpcGFydFVwbG9hZFBhcnRzJyxcbl07XG5cbi8qKlxuICogSXNzdWUgdGhlIGZvdXIgcG9saWN5IHN0YXRlbWVudHMgdGhhdCBzY29wZSBhbiBJY2ViZXJnLXRhYmxlIGdyYW50XG4gKiBjb3JyZWN0bHk6IEdsdWUgYWN0aW9ucyBvbiB0aGUgdGFibGUgQVJOLCBTMyBsaXN0LWJ1Y2tldCBhY3Rpb25zXG4gKiBvbiB0aGUgYnVja2V0IEFSTiB3aXRoIGFuIGBzMzpwcmVmaXhgIGNvbmRpdGlvbiBzbyBvbmx5IHRoZVxuICogdGFibGUncyBvd24gcHJlZml4IGNhbiBiZSBsaXN0ZWQsIFMzIGJ1Y2tldC1sZXZlbCBhY3Rpb25zIHRoYXQgRE9cbiAqIE5PVCBzdXBwb3J0IHRoZSBgczM6cHJlZml4YCBjb25kaXRpb24gKGUuZy4gYEdldEJ1Y2tldExvY2F0aW9uYCxcbiAqIGBMaXN0QnVja2V0TXVsdGlwYXJ0VXBsb2Fkc2ApIG9uIHRoZSBidWNrZXQgQVJOIHdpdGggbm8gY29uZGl0aW9uXG4gKiDigJQgaW5jbHVkaW5nIHRoZW0gaW4gdGhlIGNvbmRpdGlvbmVkIHN0YXRlbWVudCB3b3VsZCBzaWxlbnRseSBkZW55XG4gKiB0aGVtIGF0IHJ1bnRpbWUg4oCUIGFuZCBTMyBvYmplY3QtbGV2ZWwgYWN0aW9ucyBvbiB0aGVcbiAqIGBidWNrZXQvcHJlZml4KmAgQVJOLiBSZXR1cm5zIHRoZSB0YWJsZS1hY3Rpb25zIGdyYW50IChhbnkgb2YgdGhlXG4gKiBmb3VyIGlzIHN1ZmZpY2llbnQgZm9yIHRoZSBgR3JhbnRgIEFQSSBjb250cmFjdDsgdGhlIHJlc3QgYXR0YWNoXG4gKiBhcyBzaWRlIGVmZmVjdHMpLlxuICpcbiAqIEBpbnRlcm5hbFxuICovXG5leHBvcnQgZnVuY3Rpb24gZ3JhbnRTcGxpdChcbiAgICBncmFudGVlOiBJR3JhbnRhYmxlLFxuICAgIGFyZ3M6IHtcbiAgICAgICAgdGFibGVBcm46IHN0cmluZztcbiAgICAgICAgYnVja2V0QXJuOiBzdHJpbmc7XG4gICAgICAgIG9iamVjdEFybjogc3RyaW5nO1xuICAgICAgICBwcmVmaXhHbG9iOiBzdHJpbmc7XG4gICAgICAgIHRhYmxlQWN0aW9uczogc3RyaW5nW107XG4gICAgICAgIGxpc3RBY3Rpb25zOiBzdHJpbmdbXTtcbiAgICAgICAgYnVja2V0QWN0aW9uczogc3RyaW5nW107XG4gICAgICAgIG9iamVjdEFjdGlvbnM6IHN0cmluZ1tdO1xuICAgIH0sXG4pOiBHcmFudCB7XG4gICAgY29uc3QgdGFibGVHcmFudCA9IEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgZ3JhbnRlZSxcbiAgICAgICAgYWN0aW9uczogYXJncy50YWJsZUFjdGlvbnMsXG4gICAgICAgIHJlc291cmNlQXJuczogW1xuICAgICAgICAgICAgYXJncy50YWJsZUFybixcbiAgICAgICAgXSxcbiAgICB9KTtcbiAgICBpZiAoYXJncy5saXN0QWN0aW9ucy5sZW5ndGggPiAwKSB7XG4gICAgICAgIEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgICAgIGdyYW50ZWUsXG4gICAgICAgICAgICBhY3Rpb25zOiBhcmdzLmxpc3RBY3Rpb25zLFxuICAgICAgICAgICAgcmVzb3VyY2VBcm5zOiBbXG4gICAgICAgICAgICAgICAgYXJncy5idWNrZXRBcm4sXG4gICAgICAgICAgICBdLFxuICAgICAgICAgICAgY29uZGl0aW9uczoge1xuICAgICAgICAgICAgICAgIFN0cmluZ0xpa2U6IHtcbiAgICAgICAgICAgICAgICAgICAgJ3MzOnByZWZpeCc6IFtcbiAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3MucHJlZml4R2xvYixcbiAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3MucHJlZml4R2xvYi5yZXBsYWNlKC9cXCokLywgJycpLFxuICAgICAgICAgICAgICAgICAgICBdLFxuICAgICAgICAgICAgICAgIH0sXG4gICAgICAgICAgICB9LFxuICAgICAgICB9KTtcbiAgICB9XG4gICAgaWYgKGFyZ3MuYnVja2V0QWN0aW9ucy5sZW5ndGggPiAwKSB7XG4gICAgICAgIEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgICAgIGdyYW50ZWUsXG4gICAgICAgICAgICBhY3Rpb25zOiBhcmdzLmJ1Y2tldEFjdGlvbnMsXG4gICAgICAgICAgICByZXNvdXJjZUFybnM6IFtcbiAgICAgICAgICAgICAgICBhcmdzLmJ1Y2tldEFybixcbiAgICAgICAgICAgIF0sXG4gICAgICAgIH0pO1xuICAgIH1cbiAgICBHcmFudC5hZGRUb1ByaW5jaXBhbCh7XG4gICAgICAgIGdyYW50ZWUsXG4gICAgICAgIGFjdGlvbnM6IGFyZ3Mub2JqZWN0QWN0aW9ucyxcbiAgICAgICAgcmVzb3VyY2VBcm5zOiBbXG4gICAgICAgICAgICBhcmdzLm9iamVjdEFybixcbiAgICAgICAgXSxcbiAgICB9KTtcbiAgICByZXR1cm4gdGFibGVHcmFudDtcbn1cbiJdfQ==
@@ -0,0 +1,41 @@
1
+ import { CfnTable } from 'aws-cdk-lib/aws-glue';
2
+ import { IcebergType } from './iceberg-type';
3
+ import { IcebergDataFormat, IcebergFormatVersion } from './iceberg-table-properties';
4
+ import { IcebergColumn, IcebergPartitionField, IcebergSortField, IcebergTableProps } from './iceberg-table';
5
+ /** @internal */
6
+ export declare function validateProps(props: IcebergTableProps): void;
7
+ /** @internal */
8
+ export declare function normalizeLocation(location: string): string;
9
+ /** @internal */
10
+ export declare function parseS3Uri(uri: string): {
11
+ bucket: string;
12
+ key: string;
13
+ };
14
+ interface RenderedColumn {
15
+ readonly id: number;
16
+ readonly type: IcebergType;
17
+ }
18
+ /** @internal */
19
+ export interface RenderedSchema {
20
+ readonly fields: CfnTable.IcebergStructFieldProperty[];
21
+ readonly columnByName: Map<string, RenderedColumn>;
22
+ }
23
+ /** @internal */
24
+ export declare function renderSchema(columns: IcebergColumn[]): RenderedSchema;
25
+ /** @internal */
26
+ export declare function validatePartitionSpec(partitionSpec: IcebergPartitionField[] | undefined, columnByName: Map<string, RenderedColumn>): void;
27
+ /** @internal */
28
+ export declare function validateSortOrder(sortOrder: IcebergSortField[] | undefined, columnByName: Map<string, RenderedColumn>): void;
29
+ /** @internal */
30
+ export declare function renderPartitionSpec(partitionSpec: IcebergPartitionField[] | undefined, columnByName: Map<string, RenderedColumn>): CfnTable.IcebergPartitionSpecProperty | undefined;
31
+ /** @internal */
32
+ export declare function renderSortOrder(sortOrder: IcebergSortField[] | undefined, columnByName: Map<string, RenderedColumn>): CfnTable.IcebergSortOrderProperty | undefined;
33
+ /** @internal */
34
+ export declare function resolveIdentifierFieldIds(identifierFieldNames: string[] | undefined, columnByName: Map<string, RenderedColumn>): number[] | undefined;
35
+ /** @internal */
36
+ export declare function mergeProperties(dataFormat: IcebergDataFormat, formatVersion: IcebergFormatVersion, user: {
37
+ [key: string]: string;
38
+ } | undefined, comment: string | undefined): {
39
+ [key: string]: string;
40
+ };
41
+ export {};