cdk-glue-iceberg-table 0.1.6 → 0.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +83 -32
- package/dist/lib/iceberg/iceberg-table-grants.d.ts +33 -0
- package/dist/lib/iceberg/iceberg-table-grants.js +112 -0
- package/dist/lib/iceberg/iceberg-table-render.d.ts +41 -0
- package/dist/lib/iceberg/iceberg-table-render.js +241 -0
- package/dist/lib/iceberg/iceberg-table.d.ts +3 -0
- package/dist/lib/iceberg/iceberg-table.js +52 -366
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -76,6 +76,13 @@ This repo is **both** the published package and a CDK demo app:
|
|
|
76
76
|
|
|
77
77
|
- `lib/iceberg/` — the published package (`cdk-glue-iceberg-table` on npm).
|
|
78
78
|
- `lib/arceus-stack.ts`, `lib/iceberg-evolution-stack.ts`, `bin/`, `scripts/` — a CDK app that dogfoods the construct against a real AWS account, plus a bash harness that drives schema + partition evolution through real `cdk deploy`s. **Repo-only, not published to npm.**
|
|
79
|
+
- `e2e-consumer/` — a standalone CDK app that depends on the **published** `cdk-glue-iceberg-table` from npm. Proves that a fresh install + import + `cdk synth` works for downstream consumers. Runs on every PR via the `e2e-consumer` job in `.github/workflows/ci.yml`. Its `lib/surface-reference.ts` touches every exported symbol so that a rename in the published surface breaks CI. The pin in `e2e-consumer/package-lock.json` tracks the version most recently published to npm; CLAUDE.md asks for it to be bumped after each release.
|
|
80
|
+
|
|
81
|
+
How the test gates fit together:
|
|
82
|
+
|
|
83
|
+
- **`ci.yml`** runs on every PR — lint, unit tests with the 95% coverage gate, `npm pack`, and the `e2e-consumer` synth against the pinned published npm version.
|
|
84
|
+
- **`integ-test.yml`** is the real-AWS gate (`scripts/integration-test-evolution.sh` driving four `cdk deploy`s). Gated by `run-integ-test` label or `/run-integ-test` collaborator comment. PRs that touch any file under `lib/`, `bin/arceus.ts`, `cdk.json`, or the script must show a green run before merging (see CLAUDE.md §"Integration test for construct-touching PRs"). Doc-only PRs are exempt.
|
|
85
|
+
- **`publish.yml`** runs on push to `main` — trusted-publish to npm when `package.json`'s `version` is newer than the registry.
|
|
79
86
|
|
|
80
87
|
The sections [Prerequisites](#prerequisites), [Quickstart](#quickstart),
|
|
81
88
|
[Demo tables](#demo-tables-deployed-by-arceusstack), and
|
|
@@ -92,11 +99,26 @@ Before running the quickstart you need:
|
|
|
92
99
|
2. `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` set in the
|
|
93
100
|
environment (the AWS CLI sets these automatically for most
|
|
94
101
|
profile setups; `cdk` also populates them from the active profile).
|
|
95
|
-
3. `
|
|
96
|
-
in this account
|
|
97
|
-
admin and grants
|
|
98
|
-
|
|
99
|
-
can't resolve
|
|
102
|
+
3. `PRINCIPAL_ARN` set to the **ARN of an existing IAM principal**
|
|
103
|
+
in this account (IAM user, role, or federated identity). The
|
|
104
|
+
stack adds that principal as a Lake Formation admin and grants
|
|
105
|
+
it per-table `SELECT/INSERT/DELETE/ALTER/DESCRIBE` on the demo
|
|
106
|
+
Iceberg tables. Without it the deploy fails when LF can't resolve
|
|
107
|
+
the principal. The same ARN must also be the identity running
|
|
108
|
+
`cdk deploy` and any subsequent Athena queries. Local devs
|
|
109
|
+
typically set this to their IAM user ARN; CI (`integ-test.yml`)
|
|
110
|
+
sets it to the OIDC role ARN.
|
|
111
|
+
- **Only one principal is privileged at a time.** `ArceusStack`
|
|
112
|
+
writes the Lake Formation admin list with REPLACE semantics,
|
|
113
|
+
so the most recent `cdk deploy` wins. A local `cdk deploy` with
|
|
114
|
+
your user ARN revokes the OIDC role's LF grants (and breaks the
|
|
115
|
+
next integ-test run until CI redeploys), and vice versa.
|
|
116
|
+
Coordinate accordingly.
|
|
117
|
+
- **SSO / `aws-vault` users:** set `PRINCIPAL_ARN` to the
|
|
118
|
+
canonical role ARN (`arn:aws:iam::<acct>:role/<RoleName>`), not
|
|
119
|
+
the per-session `arn:aws:sts::<acct>:assumed-role/...` you'd
|
|
120
|
+
get from `aws sts get-caller-identity`. LF grants on the
|
|
121
|
+
session-suffixed ARN go stale at the next SSO refresh.
|
|
100
122
|
4. The Lake Formation service-linked role
|
|
101
123
|
`AWSServiceRoleForLakeFormationDataAccess` must exist in the
|
|
102
124
|
account. Create it once with
|
|
@@ -109,31 +131,41 @@ Before running the quickstart you need:
|
|
|
109
131
|
|
|
110
132
|
Repo source paths (npm consumers import from the package root and get the same exports under `dist/lib/iceberg/`):
|
|
111
133
|
|
|
112
|
-
- **`lib/iceberg/iceberg-table.ts
|
|
113
|
-
- **`lib/iceberg/iceberg-type.ts
|
|
114
|
-
- **`lib/iceberg/iceberg-partition-transform.ts
|
|
115
|
-
- **`lib/iceberg/iceberg-table-properties.ts
|
|
116
|
-
- **`lib/arceus-stack.ts
|
|
117
|
-
- **`lib/iceberg-evolution-stack.ts`** + **`scripts/integration-test-evolution.sh
|
|
134
|
+
- **`lib/iceberg/iceberg-table.ts`**: the `IcebergTable` L2 construct.
|
|
135
|
+
- **`lib/iceberg/iceberg-type.ts`**: `IcebergType` with primitives + `list` / `map` / `struct` factories. Renders to the JSON shape Glue's `IcebergStructField.type` expects.
|
|
136
|
+
- **`lib/iceberg/iceberg-partition-transform.ts`**: `IcebergPartitionTransform` (identity / bucket(N) / truncate(W) / year / month / day / hour / void). Each transform validates against the source column type at synth time.
|
|
137
|
+
- **`lib/iceberg/iceberg-table-properties.ts`**: `IcebergDataFormat` (parquet/orc/avro, default parquet), `IcebergFormatVersion` (v1/v2, default v2), and a validator that catches misconfigured `tableProperties` before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).
|
|
138
|
+
- **`lib/arceus-stack.ts`**: the demo stack: KMS-encrypted data lake bucket, Athena results bucket, Glue database, three demo Iceberg tables (`orders`, `events`, `customers`). Repo-only, not in the npm tarball.
|
|
139
|
+
- **`lib/iceberg-evolution-stack.ts`** + **`scripts/integration-test-evolution.sh`**: a parameterized stack and a bash harness that drives four real `cdk deploy`s to prove schema/partition evolution works end-to-end. Repo-only, not in the npm tarball.
|
|
118
140
|
|
|
119
141
|
## Quickstart
|
|
120
142
|
|
|
121
143
|
```bash
|
|
122
|
-
#
|
|
123
|
-
#
|
|
124
|
-
#
|
|
125
|
-
#
|
|
126
|
-
#
|
|
127
|
-
#
|
|
128
|
-
|
|
144
|
+
# PRINCIPAL_ARN is the ARN of the IAM principal (user, role, or
|
|
145
|
+
# federated identity) that the stack should make a Lake Formation
|
|
146
|
+
# admin and per-table grantee. It must equal the identity running
|
|
147
|
+
# `cdk deploy` and any subsequent Athena queries — otherwise the
|
|
148
|
+
# integration script's INSERT/SELECT calls fail with `Principal does
|
|
149
|
+
# not have any privilege on specified resource`.
|
|
150
|
+
#
|
|
151
|
+
# For a direct IAM-user session, the line below returns the right ARN
|
|
152
|
+
# (`arn:aws:iam::<acct>:user/<name>`).
|
|
153
|
+
#
|
|
154
|
+
# For SSO / aws-vault / any assumed-role session this returns
|
|
155
|
+
# `arn:aws:sts::<acct>:assumed-role/<RoleName>/<SessionName>`, which
|
|
156
|
+
# Lake Formation accepts but stales on the next session refresh
|
|
157
|
+
# because the SessionName changes. Set PRINCIPAL_ARN to the canonical
|
|
158
|
+
# `arn:aws:iam::<acct>:role/<RoleName>` instead — for example:
|
|
159
|
+
# export PRINCIPAL_ARN="arn:aws:iam::123456789012:role/MyDevRole"
|
|
160
|
+
export PRINCIPAL_ARN="$(aws sts get-caller-identity --query Arn --output text)"
|
|
129
161
|
|
|
130
162
|
npm install
|
|
131
|
-
npx jest #
|
|
163
|
+
npx jest # runs the suite; coverage floor in jest.config.js
|
|
132
164
|
npx cdk deploy ArceusStack --require-approval=never
|
|
133
165
|
./scripts/integration-test-evolution.sh # add + rename + drop, via cdk only
|
|
134
166
|
```
|
|
135
167
|
|
|
136
|
-
`cdk ls` will show two stacks
|
|
168
|
+
`cdk ls` will show two stacks: `ArceusStack` (the demo data lake +
|
|
137
169
|
three Iceberg tables) and `IcebergEvolutionStack` (the evolution
|
|
138
170
|
test target driven by `scripts/integration-test-evolution.sh`).
|
|
139
171
|
Deploy only `ArceusStack` for the quickstart; the evolution stack
|
|
@@ -150,7 +182,7 @@ import {
|
|
|
150
182
|
import {
|
|
151
183
|
IcebergTable,
|
|
152
184
|
IcebergType,
|
|
153
|
-
} from '
|
|
185
|
+
} from 'cdk-glue-iceberg-table';
|
|
154
186
|
|
|
155
187
|
const db = new Database(this, 'Db', {
|
|
156
188
|
databaseName: 'analytics',
|
|
@@ -183,8 +215,8 @@ new IcebergTable(this, 'Users', {
|
|
|
183
215
|
});
|
|
184
216
|
```
|
|
185
217
|
|
|
186
|
-
A table that exercises most of the surface
|
|
187
|
-
nested types, identifier fields, table properties, removal policy.
|
|
218
|
+
A table that exercises most of the surface (partitions, sort order,
|
|
219
|
+
nested types, identifier fields, table properties, removal policy).
|
|
188
220
|
This is the exact shape `ArceusStack` uses for the `orders` demo
|
|
189
221
|
table, so the column list / partition spec / properties round-trip
|
|
190
222
|
straight to the live metadata.json below.
|
|
@@ -204,7 +236,7 @@ import {
|
|
|
204
236
|
IcebergSortDirection,
|
|
205
237
|
IcebergTable,
|
|
206
238
|
IcebergType,
|
|
207
|
-
} from '
|
|
239
|
+
} from 'cdk-glue-iceberg-table';
|
|
208
240
|
|
|
209
241
|
new IcebergTable(this, 'OrdersTable', {
|
|
210
242
|
database: db,
|
|
@@ -456,7 +488,7 @@ underlying Iceberg `metadata.json` after each:
|
|
|
456
488
|
| 3 | **RENAME** `email` → `contact_email` (id 2 preserved), **ADD** partition `bucket(8)(customer_id)` | rename | + `bucket(8)(customer_id)` |
|
|
457
489
|
| 4 | **DROP** column `region` (id 4 stays retired), **DROP** partition `bucket(8)(customer_id)` | − `region` | − `customer_id_bucket` |
|
|
458
490
|
|
|
459
|
-
Last script run output (abridged
|
|
491
|
+
Last script run output (abridged: `cdk deploy` chatter and the
|
|
460
492
|
per-Athena-query state polling lines are omitted; the assertion
|
|
461
493
|
output is verbatim):
|
|
462
494
|
|
|
@@ -551,16 +583,17 @@ under `storageDescriptor`.
|
|
|
551
583
|
Even setting just `tableInput: { name: 'foo' }` next to
|
|
552
584
|
`openTableFormatInput` returns
|
|
553
585
|
`"Table metadata is expected only via TableInput or via IcebergTableInputProperties inside OpenTableFormatInput"`.
|
|
554
|
-
The construct never emits `tableInput
|
|
586
|
+
The construct never emits `tableInput`; the table-level comment goes
|
|
555
587
|
into `tableProperties['comment']`, which lives inside
|
|
556
588
|
`icebergTableInput.properties`.
|
|
557
589
|
|
|
558
|
-
(There is a third footgun
|
|
590
|
+
(There is a third footgun, field-id reuse after a column drop, that
|
|
559
591
|
the construct does **not** prevent. See the next section.)
|
|
560
592
|
|
|
561
593
|
## Known limitations
|
|
562
594
|
|
|
563
595
|
- **Field-id reuse is not detected across deploys.** If you drop a column with `id = 5` and then add a different column with `id = 5` in a later deploy, Glue accepts the UPDATE and Iceberg's metadata silently violates the "never reuse a retired id" invariant. Readers projecting old snapshots will surface deleted data under the new field's name. The construct enforces uniqueness **within one deploy** (`duplicate column id N` validator), but it doesn't compare against the live table state. The safe workflow is to always pin `id` explicitly and treat dropped ids as retired forever; never let CDK reassign an id that has ever been used.
|
|
596
|
+
- **Partition field ids are positional and not pinnable.** The construct allocates partition `fieldId` densely from 1000 in the order partitions appear in `partitionSpec`. Reordering the array across deploys reassigns those ids for unchanged logical partitions, which is the partition-spec analog of the column-id-reuse footgun above. There is no `IcebergPartitionField.fieldId` pinning prop today. The safe workflow is append-only: add new partition fields at the end of `partitionSpec`, and only drop the trailing ones.
|
|
564
597
|
- **CREATE-only metadata operation.** The CFN `IcebergInput.metadataOperation` only accepts `CREATE`; the construct always emits that. Subsequent deploys use Glue's normal `UpdateTable` path, which writes new Iceberg metadata in-place.
|
|
565
598
|
- **Format version is immutable after CREATE.** The `formatVersion` prop is read once at table creation; changing it later requires a destroy + recreate.
|
|
566
599
|
- **`merge-on-read` requires v2.** The construct rejects `write.{delete,update,merge}.mode = merge-on-read` on a v1 table at synth time.
|
|
@@ -576,7 +609,7 @@ npx jest # runs every suite under test/, prints coverage at the end
|
|
|
576
609
|
Coverage is gated at 95% statements / 95% branches / 95% lines / 95% functions
|
|
577
610
|
on `lib/**/*.ts` via the `coverageThreshold.global` block in `jest.config.js`.
|
|
578
611
|
A failing gate fails the suite, so the README does not paste a transcript that
|
|
579
|
-
would drift after the next refactor
|
|
612
|
+
would drift after the next refactor; run the command locally for the live
|
|
580
613
|
numbers.
|
|
581
614
|
|
|
582
615
|
## Project layout
|
|
@@ -592,7 +625,7 @@ arceus/
|
|
|
592
625
|
│ ├── iceberg-type.ts # IcebergType + struct/list/map/decimal/fixed
|
|
593
626
|
│ ├── iceberg-partition-transform.ts
|
|
594
627
|
│ ├── iceberg-table-properties.ts # Format/version enums + property validation
|
|
595
|
-
│ └── index.ts # Re-exports
|
|
628
|
+
│ └── index.ts # Re-exports (the npm package's entry point)
|
|
596
629
|
├── test/
|
|
597
630
|
│ ├── arceus-stack.test.ts
|
|
598
631
|
│ ├── iceberg-evolution-stack.test.ts
|
|
@@ -601,11 +634,29 @@ arceus/
|
|
|
601
634
|
│ ├── iceberg-table-properties.test.ts
|
|
602
635
|
│ ├── iceberg-table.test.ts
|
|
603
636
|
│ └── iceberg-type.test.ts
|
|
637
|
+
├── e2e-consumer/ # Standalone CDK app that consumes the
|
|
638
|
+
│ │ # published npm package — proves a fresh
|
|
639
|
+
│ │ # install + import + synth works. Runs
|
|
640
|
+
│ │ # on every PR via the `e2e-consumer` job
|
|
641
|
+
│ │ # in `.github/workflows/ci.yml`.
|
|
642
|
+
│ ├── bin/app.ts
|
|
643
|
+
│ ├── lib/consumer-stack.ts # Realistic consumer (one IcebergTable)
|
|
644
|
+
│ └── lib/surface-reference.ts # Anchors every exported symbol so a
|
|
645
|
+
│ # rename in the published surface breaks CI
|
|
604
646
|
├── scripts/
|
|
605
647
|
│ └── integration-test-evolution.sh # End-to-end evolution harness
|
|
648
|
+
├── docs/
|
|
649
|
+
│ └── integ-test-setup.md # AWS-side prerequisites for the integ-test
|
|
650
|
+
│ # workflow (OIDC provider, IAM role, repo var)
|
|
651
|
+
├── .github/workflows/
|
|
652
|
+
│ ├── ci.yml # Lint + test + build + pack + e2e-consumer
|
|
653
|
+
│ ├── publish.yml # Trusted-publish to npm on version bump
|
|
654
|
+
│ └── integ-test.yml # Real-AWS evolution test (gated by label /
|
|
655
|
+
│ # `/run-integ-test` comment)
|
|
606
656
|
├── cdk.json
|
|
607
|
-
├── package.json
|
|
657
|
+
├── package.json # The published package's manifest
|
|
658
|
+
├── tsconfig.json # Wide include (used by ESLint and dev)
|
|
659
|
+
├── tsconfig.build.json # Narrow include (used by `npm publish`)
|
|
608
660
|
├── jest.config.js
|
|
609
|
-
|
|
610
|
-
└── tsconfig.json
|
|
661
|
+
└── eslint.config.js
|
|
611
662
|
```
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
import { Grant, IGrantable } from 'aws-cdk-lib/aws-iam';
|
|
2
|
+
export declare const READ_TABLE_ACTIONS: string[];
|
|
3
|
+
export declare const WRITE_TABLE_ACTIONS: string[];
|
|
4
|
+
export declare const READ_S3_LIST_ACTIONS: string[];
|
|
5
|
+
export declare const READ_S3_BUCKET_ACTIONS: string[];
|
|
6
|
+
export declare const WRITE_S3_BUCKET_ACTIONS: string[];
|
|
7
|
+
export declare const READ_S3_OBJECT_ACTIONS: string[];
|
|
8
|
+
export declare const WRITE_S3_OBJECT_ACTIONS: string[];
|
|
9
|
+
/**
|
|
10
|
+
* Issue the four policy statements that scope an Iceberg-table grant
|
|
11
|
+
* correctly: Glue actions on the table ARN, S3 list-bucket actions
|
|
12
|
+
* on the bucket ARN with an `s3:prefix` condition so only the
|
|
13
|
+
* table's own prefix can be listed, S3 bucket-level actions that DO
|
|
14
|
+
* NOT support the `s3:prefix` condition (e.g. `GetBucketLocation`,
|
|
15
|
+
* `ListBucketMultipartUploads`) on the bucket ARN with no condition
|
|
16
|
+
* — including them in the conditioned statement would silently deny
|
|
17
|
+
* them at runtime — and S3 object-level actions on the
|
|
18
|
+
* `bucket/prefix*` ARN. Returns the table-actions grant (any of the
|
|
19
|
+
* four is sufficient for the `Grant` API contract; the rest attach
|
|
20
|
+
* as side effects).
|
|
21
|
+
*
|
|
22
|
+
* @internal
|
|
23
|
+
*/
|
|
24
|
+
export declare function grantSplit(grantee: IGrantable, args: {
|
|
25
|
+
tableArn: string;
|
|
26
|
+
bucketArn: string;
|
|
27
|
+
objectArn: string;
|
|
28
|
+
prefixGlob: string;
|
|
29
|
+
tableActions: string[];
|
|
30
|
+
listActions: string[];
|
|
31
|
+
bucketActions: string[];
|
|
32
|
+
objectActions: string[];
|
|
33
|
+
}): Grant;
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.WRITE_S3_OBJECT_ACTIONS = exports.READ_S3_OBJECT_ACTIONS = exports.WRITE_S3_BUCKET_ACTIONS = exports.READ_S3_BUCKET_ACTIONS = exports.READ_S3_LIST_ACTIONS = exports.WRITE_TABLE_ACTIONS = exports.READ_TABLE_ACTIONS = void 0;
|
|
4
|
+
exports.grantSplit = grantSplit;
|
|
5
|
+
const aws_iam_1 = require("aws-cdk-lib/aws-iam");
|
|
6
|
+
/// /////////////////////////////////////////////////
|
|
7
|
+
/// /////////////////////////////////////////////////
|
|
8
|
+
// Grant action sets
|
|
9
|
+
exports.READ_TABLE_ACTIONS = [
|
|
10
|
+
'glue:BatchGetPartition',
|
|
11
|
+
'glue:GetPartition',
|
|
12
|
+
'glue:GetPartitions',
|
|
13
|
+
'glue:GetTable',
|
|
14
|
+
'glue:GetTables',
|
|
15
|
+
'glue:GetTableVersion',
|
|
16
|
+
'glue:GetTableVersions',
|
|
17
|
+
];
|
|
18
|
+
exports.WRITE_TABLE_ACTIONS = [
|
|
19
|
+
'glue:BatchCreatePartition',
|
|
20
|
+
'glue:BatchDeletePartition',
|
|
21
|
+
'glue:CreatePartition',
|
|
22
|
+
'glue:DeletePartition',
|
|
23
|
+
'glue:UpdatePartition',
|
|
24
|
+
'glue:UpdateTable',
|
|
25
|
+
];
|
|
26
|
+
/// S3 actions that operate at the bucket level AND support the
|
|
27
|
+
/// `s3:prefix` request condition key (per the S3 docs). Granted on
|
|
28
|
+
/// the bucket ARN with a `StringLike s3:prefix = [<prefix>*, <prefix>]`
|
|
29
|
+
/// condition so the grantee can only list the table's own prefix.
|
|
30
|
+
exports.READ_S3_LIST_ACTIONS = [
|
|
31
|
+
's3:ListBucket',
|
|
32
|
+
];
|
|
33
|
+
/// S3 actions that operate at the bucket level but DO NOT support
|
|
34
|
+
/// `s3:prefix`. Granted on the bucket ARN with no condition — adding
|
|
35
|
+
/// one would silently deny these actions at runtime even though they
|
|
36
|
+
/// appear in the policy document.
|
|
37
|
+
exports.READ_S3_BUCKET_ACTIONS = [
|
|
38
|
+
's3:GetBucketLocation',
|
|
39
|
+
];
|
|
40
|
+
exports.WRITE_S3_BUCKET_ACTIONS = [
|
|
41
|
+
's3:ListBucketMultipartUploads',
|
|
42
|
+
];
|
|
43
|
+
/// S3 actions that operate at the object level. Must be granted on
|
|
44
|
+
/// the `bucket/prefix*` ARN.
|
|
45
|
+
exports.READ_S3_OBJECT_ACTIONS = [
|
|
46
|
+
's3:GetObject',
|
|
47
|
+
];
|
|
48
|
+
exports.WRITE_S3_OBJECT_ACTIONS = [
|
|
49
|
+
's3:PutObject',
|
|
50
|
+
's3:DeleteObject',
|
|
51
|
+
's3:AbortMultipartUpload',
|
|
52
|
+
's3:ListMultipartUploadParts',
|
|
53
|
+
];
|
|
54
|
+
/**
|
|
55
|
+
* Issue the four policy statements that scope an Iceberg-table grant
|
|
56
|
+
* correctly: Glue actions on the table ARN, S3 list-bucket actions
|
|
57
|
+
* on the bucket ARN with an `s3:prefix` condition so only the
|
|
58
|
+
* table's own prefix can be listed, S3 bucket-level actions that DO
|
|
59
|
+
* NOT support the `s3:prefix` condition (e.g. `GetBucketLocation`,
|
|
60
|
+
* `ListBucketMultipartUploads`) on the bucket ARN with no condition
|
|
61
|
+
* — including them in the conditioned statement would silently deny
|
|
62
|
+
* them at runtime — and S3 object-level actions on the
|
|
63
|
+
* `bucket/prefix*` ARN. Returns the table-actions grant (any of the
|
|
64
|
+
* four is sufficient for the `Grant` API contract; the rest attach
|
|
65
|
+
* as side effects).
|
|
66
|
+
*
|
|
67
|
+
* @internal
|
|
68
|
+
*/
|
|
69
|
+
function grantSplit(grantee, args) {
|
|
70
|
+
const tableGrant = aws_iam_1.Grant.addToPrincipal({
|
|
71
|
+
grantee,
|
|
72
|
+
actions: args.tableActions,
|
|
73
|
+
resourceArns: [
|
|
74
|
+
args.tableArn,
|
|
75
|
+
],
|
|
76
|
+
});
|
|
77
|
+
if (args.listActions.length > 0) {
|
|
78
|
+
aws_iam_1.Grant.addToPrincipal({
|
|
79
|
+
grantee,
|
|
80
|
+
actions: args.listActions,
|
|
81
|
+
resourceArns: [
|
|
82
|
+
args.bucketArn,
|
|
83
|
+
],
|
|
84
|
+
conditions: {
|
|
85
|
+
StringLike: {
|
|
86
|
+
's3:prefix': [
|
|
87
|
+
args.prefixGlob,
|
|
88
|
+
args.prefixGlob.replace(/\*$/, ''),
|
|
89
|
+
],
|
|
90
|
+
},
|
|
91
|
+
},
|
|
92
|
+
});
|
|
93
|
+
}
|
|
94
|
+
if (args.bucketActions.length > 0) {
|
|
95
|
+
aws_iam_1.Grant.addToPrincipal({
|
|
96
|
+
grantee,
|
|
97
|
+
actions: args.bucketActions,
|
|
98
|
+
resourceArns: [
|
|
99
|
+
args.bucketArn,
|
|
100
|
+
],
|
|
101
|
+
});
|
|
102
|
+
}
|
|
103
|
+
aws_iam_1.Grant.addToPrincipal({
|
|
104
|
+
grantee,
|
|
105
|
+
actions: args.objectActions,
|
|
106
|
+
resourceArns: [
|
|
107
|
+
args.objectArn,
|
|
108
|
+
],
|
|
109
|
+
});
|
|
110
|
+
return tableGrant;
|
|
111
|
+
}
|
|
112
|
+
//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiaWNlYmVyZy10YWJsZS1ncmFudHMuanMiLCJzb3VyY2VSb290IjoiIiwic291cmNlcyI6WyIuLi8uLi8uLi9saWIvaWNlYmVyZy9pY2ViZXJnLXRhYmxlLWdyYW50cy50cyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOzs7QUE0RUEsZ0NBc0RDO0FBbElELGlEQUc2QjtBQUU3QixxREFBcUQ7QUFDckQscURBQXFEO0FBQ3JELG9CQUFvQjtBQUVQLFFBQUEsa0JBQWtCLEdBQUc7SUFDOUIsd0JBQXdCO0lBQ3hCLG1CQUFtQjtJQUNuQixvQkFBb0I7SUFDcEIsZUFBZTtJQUNmLGdCQUFnQjtJQUNoQixzQkFBc0I7SUFDdEIsdUJBQXVCO0NBQzFCLENBQUM7QUFFVyxRQUFBLG1CQUFtQixHQUFHO0lBQy9CLDJCQUEyQjtJQUMzQiwyQkFBMkI7SUFDM0Isc0JBQXNCO0lBQ3RCLHNCQUFzQjtJQUN0QixzQkFBc0I7SUFDdEIsa0JBQWtCO0NBQ3JCLENBQUM7QUFFRiwrREFBK0Q7QUFDL0QsbUVBQW1FO0FBQ25FLHdFQUF3RTtBQUN4RSxrRUFBa0U7QUFDckQsUUFBQSxvQkFBb0IsR0FBRztJQUNoQyxlQUFlO0NBQ2xCLENBQUM7QUFFRixrRUFBa0U7QUFDbEUscUVBQXFFO0FBQ3JFLHFFQUFxRTtBQUNyRSxrQ0FBa0M7QUFDckIsUUFBQSxzQkFBc0IsR0FBRztJQUNsQyxzQkFBc0I7Q0FDekIsQ0FBQztBQUVXLFFBQUEsdUJBQXVCLEdBQUc7SUFDbkMsK0JBQStCO0NBQ2xDLENBQUM7QUFFRixtRUFBbUU7QUFDbkUsNkJBQTZCO0FBQ2hCLFFBQUEsc0JBQXNCLEdBQUc7SUFDbEMsY0FBYztDQUNqQixDQUFDO0FBRVcsUUFBQSx1QkFBdUIsR0FBRztJQUNuQyxjQUFjO0lBQ2QsaUJBQWlCO0lBQ2pCLHlCQUF5QjtJQUN6Qiw2QkFBNkI7Q0FDaEMsQ0FBQztBQUVGOzs7Ozs7Ozs7Ozs7OztHQWNHO0FBQ0gsU0FBZ0IsVUFBVSxDQUN0QixPQUFtQixFQUNuQixJQVNDO0lBRUQsTUFBTSxVQUFVLEdBQUcsZUFBSyxDQUFDLGNBQWMsQ0FBQztRQUNwQyxPQUFPO1FBQ1AsT0FBTyxFQUFFLElBQUksQ0FBQyxZQUFZO1FBQzFCLFlBQVksRUFBRTtZQUNWLElBQUksQ0FBQyxRQUFRO1NBQ2hCO0tBQ0osQ0FBQyxDQUFDO0lBQ0gsSUFBSSxJQUFJLENBQUMsV0FBVyxDQUFDLE1BQU0sR0FBRyxDQUFDLEVBQUUsQ0FBQztRQUM5QixlQUFLLENBQUMsY0FBYyxDQUFDO1lBQ2pCLE9BQU87WUFDUCxPQUFPLEVBQUUsSUFBSSxDQUFDLFdBQVc7WUFDekIsWUFBWSxFQUFFO2dCQUNWLElBQUksQ0FBQyxTQUFTO2FBQ2pCO1lBQ0QsVUFBVSxFQUFFO2dCQUNSLFVBQVUsRUFBRTtvQkFDUixXQUFXLEVBQUU7d0JBQ1QsSUFBSSxDQUFDLFVBQVU7d0JBQ2YsSUFBSSxDQUFDLFVBQVUsQ0FBQyxPQUFPLENBQUMsS0FBSyxFQUFFLEVBQUUsQ0FBQztxQkFDckM7aUJBQ0o7YUFDSjtTQUNKLENBQUMsQ0FBQztJQUNQLENBQUM7SUFDRCxJQUFJLElBQUksQ0FBQyxhQUFhLENBQUMsTUFBTSxHQUFHLENBQUMsRUFBRSxDQUFDO1FBQ2hDLGVBQUssQ0FBQyxjQUFjLENBQUM7WUFDakIsT0FBTztZQUNQLE9BQU8sRUFBRSxJQUFJLENBQUMsYUFBYTtZQUMzQixZQUFZLEVBQUU7Z0JBQ1YsSUFBSSxDQUFDLFNBQVM7YUFDakI7U0FDSixDQUFDLENBQUM7SUFDUCxDQUFDO0lBQ0QsZUFBSyxDQUFDLGNBQWMsQ0FBQztRQUNqQixPQUFPO1FBQ1AsT0FBTyxFQUFFLElBQUksQ0FBQyxhQUFhO1FBQzNCLFlBQVksRUFBRTtZQUNWLElBQUksQ0FBQyxTQUFTO1NBQ2pCO0tBQ0osQ0FBQyxDQUFDO0lBQ0gsT0FBTyxVQUFVLENBQUM7QUFDdEIsQ0FBQyIsInNvdXJjZXNDb250ZW50IjpbImltcG9ydCB7XG4gICAgR3JhbnQsXG4gICAgSUdyYW50YWJsZSxcbn0gZnJvbSAnYXdzLWNkay1saWIvYXdzLWlhbSc7XG5cbi8vLyAvLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vXG4vLy8gLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vL1xuLy8gR3JhbnQgYWN0aW9uIHNldHNcblxuZXhwb3J0IGNvbnN0IFJFQURfVEFCTEVfQUNUSU9OUyA9IFtcbiAgICAnZ2x1ZTpCYXRjaEdldFBhcnRpdGlvbicsXG4gICAgJ2dsdWU6R2V0UGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpHZXRQYXJ0aXRpb25zJyxcbiAgICAnZ2x1ZTpHZXRUYWJsZScsXG4gICAgJ2dsdWU6R2V0VGFibGVzJyxcbiAgICAnZ2x1ZTpHZXRUYWJsZVZlcnNpb24nLFxuICAgICdnbHVlOkdldFRhYmxlVmVyc2lvbnMnLFxuXTtcblxuZXhwb3J0IGNvbnN0IFdSSVRFX1RBQkxFX0FDVElPTlMgPSBbXG4gICAgJ2dsdWU6QmF0Y2hDcmVhdGVQYXJ0aXRpb24nLFxuICAgICdnbHVlOkJhdGNoRGVsZXRlUGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpDcmVhdGVQYXJ0aXRpb24nLFxuICAgICdnbHVlOkRlbGV0ZVBhcnRpdGlvbicsXG4gICAgJ2dsdWU6VXBkYXRlUGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpVcGRhdGVUYWJsZScsXG5dO1xuXG4vLy8gUzMgYWN0aW9ucyB0aGF0IG9wZXJhdGUgYXQgdGhlIGJ1Y2tldCBsZXZlbCBBTkQgc3VwcG9ydCB0aGVcbi8vLyBgczM6cHJlZml4YCByZXF1ZXN0IGNvbmRpdGlvbiBrZXkgKHBlciB0aGUgUzMgZG9jcykuIEdyYW50ZWQgb25cbi8vLyB0aGUgYnVja2V0IEFSTiB3aXRoIGEgYFN0cmluZ0xpa2UgczM6cHJlZml4ID0gWzxwcmVmaXg+KiwgPHByZWZpeD5dYFxuLy8vIGNvbmRpdGlvbiBzbyB0aGUgZ3JhbnRlZSBjYW4gb25seSBsaXN0IHRoZSB0YWJsZSdzIG93biBwcmVmaXguXG5leHBvcnQgY29uc3QgUkVBRF9TM19MSVNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOkxpc3RCdWNrZXQnLFxuXTtcblxuLy8vIFMzIGFjdGlvbnMgdGhhdCBvcGVyYXRlIGF0IHRoZSBidWNrZXQgbGV2ZWwgYnV0IERPIE5PVCBzdXBwb3J0XG4vLy8gYHMzOnByZWZpeGAuIEdyYW50ZWQgb24gdGhlIGJ1Y2tldCBBUk4gd2l0aCBubyBjb25kaXRpb24g4oCUIGFkZGluZ1xuLy8vIG9uZSB3b3VsZCBzaWxlbnRseSBkZW55IHRoZXNlIGFjdGlvbnMgYXQgcnVudGltZSBldmVuIHRob3VnaCB0aGV5XG4vLy8gYXBwZWFyIGluIHRoZSBwb2xpY3kgZG9jdW1lbnQuXG5leHBvcnQgY29uc3QgUkVBRF9TM19CVUNLRVRfQUNUSU9OUyA9IFtcbiAgICAnczM6R2V0QnVja2V0TG9jYXRpb24nLFxuXTtcblxuZXhwb3J0IGNvbnN0IFdSSVRFX1MzX0JVQ0tFVF9BQ1RJT05TID0gW1xuICAgICdzMzpMaXN0QnVja2V0TXVsdGlwYXJ0VXBsb2FkcycsXG5dO1xuXG4vLy8gUzMgYWN0aW9ucyB0aGF0IG9wZXJhdGUgYXQgdGhlIG9iamVjdCBsZXZlbC4gTXVzdCBiZSBncmFudGVkIG9uXG4vLy8gdGhlIGBidWNrZXQvcHJlZml4KmAgQVJOLlxuZXhwb3J0IGNvbnN0IFJFQURfUzNfT0JKRUNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOkdldE9iamVjdCcsXG5dO1xuXG5leHBvcnQgY29uc3QgV1JJVEVfUzNfT0JKRUNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOlB1dE9iamVjdCcsXG4gICAgJ3MzOkRlbGV0ZU9iamVjdCcsXG4gICAgJ3MzOkFib3J0TXVsdGlwYXJ0VXBsb2FkJyxcbiAgICAnczM6TGlzdE11bHRpcGFydFVwbG9hZFBhcnRzJyxcbl07XG5cbi8qKlxuICogSXNzdWUgdGhlIGZvdXIgcG9saWN5IHN0YXRlbWVudHMgdGhhdCBzY29wZSBhbiBJY2ViZXJnLXRhYmxlIGdyYW50XG4gKiBjb3JyZWN0bHk6IEdsdWUgYWN0aW9ucyBvbiB0aGUgdGFibGUgQVJOLCBTMyBsaXN0LWJ1Y2tldCBhY3Rpb25zXG4gKiBvbiB0aGUgYnVja2V0IEFSTiB3aXRoIGFuIGBzMzpwcmVmaXhgIGNvbmRpdGlvbiBzbyBvbmx5IHRoZVxuICogdGFibGUncyBvd24gcHJlZml4IGNhbiBiZSBsaXN0ZWQsIFMzIGJ1Y2tldC1sZXZlbCBhY3Rpb25zIHRoYXQgRE9cbiAqIE5PVCBzdXBwb3J0IHRoZSBgczM6cHJlZml4YCBjb25kaXRpb24gKGUuZy4gYEdldEJ1Y2tldExvY2F0aW9uYCxcbiAqIGBMaXN0QnVja2V0TXVsdGlwYXJ0VXBsb2Fkc2ApIG9uIHRoZSBidWNrZXQgQVJOIHdpdGggbm8gY29uZGl0aW9uXG4gKiDigJQgaW5jbHVkaW5nIHRoZW0gaW4gdGhlIGNvbmRpdGlvbmVkIHN0YXRlbWVudCB3b3VsZCBzaWxlbnRseSBkZW55XG4gKiB0aGVtIGF0IHJ1bnRpbWUg4oCUIGFuZCBTMyBvYmplY3QtbGV2ZWwgYWN0aW9ucyBvbiB0aGVcbiAqIGBidWNrZXQvcHJlZml4KmAgQVJOLiBSZXR1cm5zIHRoZSB0YWJsZS1hY3Rpb25zIGdyYW50IChhbnkgb2YgdGhlXG4gKiBmb3VyIGlzIHN1ZmZpY2llbnQgZm9yIHRoZSBgR3JhbnRgIEFQSSBjb250cmFjdDsgdGhlIHJlc3QgYXR0YWNoXG4gKiBhcyBzaWRlIGVmZmVjdHMpLlxuICpcbiAqIEBpbnRlcm5hbFxuICovXG5leHBvcnQgZnVuY3Rpb24gZ3JhbnRTcGxpdChcbiAgICBncmFudGVlOiBJR3JhbnRhYmxlLFxuICAgIGFyZ3M6IHtcbiAgICAgICAgdGFibGVBcm46IHN0cmluZztcbiAgICAgICAgYnVja2V0QXJuOiBzdHJpbmc7XG4gICAgICAgIG9iamVjdEFybjogc3RyaW5nO1xuICAgICAgICBwcmVmaXhHbG9iOiBzdHJpbmc7XG4gICAgICAgIHRhYmxlQWN0aW9uczogc3RyaW5nW107XG4gICAgICAgIGxpc3RBY3Rpb25zOiBzdHJpbmdbXTtcbiAgICAgICAgYnVja2V0QWN0aW9uczogc3RyaW5nW107XG4gICAgICAgIG9iamVjdEFjdGlvbnM6IHN0cmluZ1tdO1xuICAgIH0sXG4pOiBHcmFudCB7XG4gICAgY29uc3QgdGFibGVHcmFudCA9IEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgZ3JhbnRlZSxcbiAgICAgICAgYWN0aW9uczogYXJncy50YWJsZUFjdGlvbnMsXG4gICAgICAgIHJlc291cmNlQXJuczogW1xuICAgICAgICAgICAgYXJncy50YWJsZUFybixcbiAgICAgICAgXSxcbiAgICB9KTtcbiAgICBpZiAoYXJncy5saXN0QWN0aW9ucy5sZW5ndGggPiAwKSB7XG4gICAgICAgIEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgICAgIGdyYW50ZWUsXG4gICAgICAgICAgICBhY3Rpb25zOiBhcmdzLmxpc3RBY3Rpb25zLFxuICAgICAgICAgICAgcmVzb3VyY2VBcm5zOiBbXG4gICAgICAgICAgICAgICAgYXJncy5idWNrZXRBcm4sXG4gICAgICAgICAgICBdLFxuICAgICAgICAgICAgY29uZGl0aW9uczoge1xuICAgICAgICAgICAgICAgIFN0cmluZ0xpa2U6IHtcbiAgICAgICAgICAgICAgICAgICAgJ3MzOnByZWZpeCc6IFtcbiAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3MucHJlZml4R2xvYixcbiAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3MucHJlZml4R2xvYi5yZXBsYWNlKC9cXCokLywgJycpLFxuICAgICAgICAgICAgICAgICAgICBdLFxuICAgICAgICAgICAgICAgIH0sXG4gICAgICAgICAgICB9LFxuICAgICAgICB9KTtcbiAgICB9XG4gICAgaWYgKGFyZ3MuYnVja2V0QWN0aW9ucy5sZW5ndGggPiAwKSB7XG4gICAgICAgIEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgICAgIGdyYW50ZWUsXG4gICAgICAgICAgICBhY3Rpb25zOiBhcmdzLmJ1Y2tldEFjdGlvbnMsXG4gICAgICAgICAgICByZXNvdXJjZUFybnM6IFtcbiAgICAgICAgICAgICAgICBhcmdzLmJ1Y2tldEFybixcbiAgICAgICAgICAgIF0sXG4gICAgICAgIH0pO1xuICAgIH1cbiAgICBHcmFudC5hZGRUb1ByaW5jaXBhbCh7XG4gICAgICAgIGdyYW50ZWUsXG4gICAgICAgIGFjdGlvbnM6IGFyZ3Mub2JqZWN0QWN0aW9ucyxcbiAgICAgICAgcmVzb3VyY2VBcm5zOiBbXG4gICAgICAgICAgICBhcmdzLm9iamVjdEFybixcbiAgICAgICAgXSxcbiAgICB9KTtcbiAgICByZXR1cm4gdGFibGVHcmFudDtcbn1cbiJdfQ==
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
import { CfnTable } from 'aws-cdk-lib/aws-glue';
|
|
2
|
+
import { IcebergType } from './iceberg-type';
|
|
3
|
+
import { IcebergDataFormat, IcebergFormatVersion } from './iceberg-table-properties';
|
|
4
|
+
import { IcebergColumn, IcebergPartitionField, IcebergSortField, IcebergTableProps } from './iceberg-table';
|
|
5
|
+
/** @internal */
|
|
6
|
+
export declare function validateProps(props: IcebergTableProps): void;
|
|
7
|
+
/** @internal */
|
|
8
|
+
export declare function normalizeLocation(location: string): string;
|
|
9
|
+
/** @internal */
|
|
10
|
+
export declare function parseS3Uri(uri: string): {
|
|
11
|
+
bucket: string;
|
|
12
|
+
key: string;
|
|
13
|
+
};
|
|
14
|
+
interface RenderedColumn {
|
|
15
|
+
readonly id: number;
|
|
16
|
+
readonly type: IcebergType;
|
|
17
|
+
}
|
|
18
|
+
/** @internal */
|
|
19
|
+
export interface RenderedSchema {
|
|
20
|
+
readonly fields: CfnTable.IcebergStructFieldProperty[];
|
|
21
|
+
readonly columnByName: Map<string, RenderedColumn>;
|
|
22
|
+
}
|
|
23
|
+
/** @internal */
|
|
24
|
+
export declare function renderSchema(columns: IcebergColumn[]): RenderedSchema;
|
|
25
|
+
/** @internal */
|
|
26
|
+
export declare function validatePartitionSpec(partitionSpec: IcebergPartitionField[] | undefined, columnByName: Map<string, RenderedColumn>): void;
|
|
27
|
+
/** @internal */
|
|
28
|
+
export declare function validateSortOrder(sortOrder: IcebergSortField[] | undefined, columnByName: Map<string, RenderedColumn>): void;
|
|
29
|
+
/** @internal */
|
|
30
|
+
export declare function renderPartitionSpec(partitionSpec: IcebergPartitionField[] | undefined, columnByName: Map<string, RenderedColumn>): CfnTable.IcebergPartitionSpecProperty | undefined;
|
|
31
|
+
/** @internal */
|
|
32
|
+
export declare function renderSortOrder(sortOrder: IcebergSortField[] | undefined, columnByName: Map<string, RenderedColumn>): CfnTable.IcebergSortOrderProperty | undefined;
|
|
33
|
+
/** @internal */
|
|
34
|
+
export declare function resolveIdentifierFieldIds(identifierFieldNames: string[] | undefined, columnByName: Map<string, RenderedColumn>): number[] | undefined;
|
|
35
|
+
/** @internal */
|
|
36
|
+
export declare function mergeProperties(dataFormat: IcebergDataFormat, formatVersion: IcebergFormatVersion, user: {
|
|
37
|
+
[key: string]: string;
|
|
38
|
+
} | undefined, comment: string | undefined): {
|
|
39
|
+
[key: string]: string;
|
|
40
|
+
};
|
|
41
|
+
export {};
|