cdk-glue-iceberg-table 0.1.7 → 0.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -19
- package/dist/lib/iceberg/iceberg-table-grants.d.ts +33 -0
- package/dist/lib/iceberg/iceberg-table-grants.js +112 -0
- package/dist/lib/iceberg/iceberg-table-render.d.ts +41 -0
- package/dist/lib/iceberg/iceberg-table-render.js +241 -0
- package/dist/lib/iceberg/iceberg-table.d.ts +3 -0
- package/dist/lib/iceberg/iceberg-table.js +52 -366
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -76,11 +76,11 @@ This repo is **both** the published package and a CDK demo app:
|
|
|
76
76
|
|
|
77
77
|
- `lib/iceberg/` — the published package (`cdk-glue-iceberg-table` on npm).
|
|
78
78
|
- `lib/arceus-stack.ts`, `lib/iceberg-evolution-stack.ts`, `bin/`, `scripts/` — a CDK app that dogfoods the construct against a real AWS account, plus a bash harness that drives schema + partition evolution through real `cdk deploy`s. **Repo-only, not published to npm.**
|
|
79
|
-
- `e2e-consumer/` — a standalone CDK app that depends on the **published** `cdk-glue-iceberg-table` from npm. Proves that a fresh install + import + `cdk synth` works for downstream consumers. Runs on every PR via the `e2e-consumer` job in `.github/workflows/ci.yml`. Its `lib/surface-reference.ts` touches every exported symbol so that a rename
|
|
79
|
+
- `e2e-consumer/` — a standalone CDK app that depends on the **published** `cdk-glue-iceberg-table` from npm. Proves that a fresh install + import + `cdk synth` works for downstream consumers. Runs on every PR via the `e2e-consumer` job in `.github/workflows/ci.yml`. Its `lib/surface-reference.ts` touches every exported symbol so that a rename in the published surface breaks CI. The pin in `e2e-consumer/package-lock.json` tracks the version most recently published to npm; CLAUDE.md asks for it to be bumped after each release.
|
|
80
80
|
|
|
81
81
|
How the test gates fit together:
|
|
82
82
|
|
|
83
|
-
- **`ci.yml`** runs on every PR — lint, unit tests with 95% coverage gate, `npm pack`, and the `e2e-consumer` synth against the
|
|
83
|
+
- **`ci.yml`** runs on every PR — lint, unit tests with the 95% coverage gate, `npm pack`, and the `e2e-consumer` synth against the pinned published npm version.
|
|
84
84
|
- **`integ-test.yml`** is the real-AWS gate (`scripts/integration-test-evolution.sh` driving four `cdk deploy`s). Gated by `run-integ-test` label or `/run-integ-test` collaborator comment. PRs that touch any file under `lib/`, `bin/arceus.ts`, `cdk.json`, or the script must show a green run before merging (see CLAUDE.md §"Integration test for construct-touching PRs"). Doc-only PRs are exempt.
|
|
85
85
|
- **`publish.yml`** runs on push to `main` — trusted-publish to npm when `package.json`'s `version` is newer than the registry.
|
|
86
86
|
|
|
@@ -100,12 +100,12 @@ Before running the quickstart you need:
|
|
|
100
100
|
environment (the AWS CLI sets these automatically for most
|
|
101
101
|
profile setups; `cdk` also populates them from the active profile).
|
|
102
102
|
3. `PRINCIPAL_ARN` set to the **ARN of an existing IAM principal**
|
|
103
|
-
in this account
|
|
103
|
+
in this account (IAM user, role, or federated identity). The
|
|
104
104
|
stack adds that principal as a Lake Formation admin and grants
|
|
105
105
|
it per-table `SELECT/INSERT/DELETE/ALTER/DESCRIBE` on the demo
|
|
106
106
|
Iceberg tables. Without it the deploy fails when LF can't resolve
|
|
107
107
|
the principal. The same ARN must also be the identity running
|
|
108
|
-
`cdk deploy` and any subsequent Athena queries
|
|
108
|
+
`cdk deploy` and any subsequent Athena queries. Local devs
|
|
109
109
|
typically set this to their IAM user ARN; CI (`integ-test.yml`)
|
|
110
110
|
sets it to the OIDC role ARN.
|
|
111
111
|
- **Only one principal is privileged at a time.** `ArceusStack`
|
|
@@ -131,12 +131,12 @@ Before running the quickstart you need:
|
|
|
131
131
|
|
|
132
132
|
Repo source paths (npm consumers import from the package root and get the same exports under `dist/lib/iceberg/`):
|
|
133
133
|
|
|
134
|
-
- **`lib/iceberg/iceberg-table.ts
|
|
135
|
-
- **`lib/iceberg/iceberg-type.ts
|
|
136
|
-
- **`lib/iceberg/iceberg-partition-transform.ts
|
|
137
|
-
- **`lib/iceberg/iceberg-table-properties.ts
|
|
138
|
-
- **`lib/arceus-stack.ts
|
|
139
|
-
- **`lib/iceberg-evolution-stack.ts`** + **`scripts/integration-test-evolution.sh
|
|
134
|
+
- **`lib/iceberg/iceberg-table.ts`**: the `IcebergTable` L2 construct.
|
|
135
|
+
- **`lib/iceberg/iceberg-type.ts`**: `IcebergType` with primitives + `list` / `map` / `struct` factories. Renders to the JSON shape Glue's `IcebergStructField.type` expects.
|
|
136
|
+
- **`lib/iceberg/iceberg-partition-transform.ts`**: `IcebergPartitionTransform` (identity / bucket(N) / truncate(W) / year / month / day / hour / void). Each transform validates against the source column type at synth time.
|
|
137
|
+
- **`lib/iceberg/iceberg-table-properties.ts`**: `IcebergDataFormat` (parquet/orc/avro, default parquet), `IcebergFormatVersion` (v1/v2, default v2), and a validator that catches misconfigured `tableProperties` before they leave your machine (wrong codec for the chosen format, `merge-on-read` on a v1 table, non-positive numeric values, …).
|
|
138
|
+
- **`lib/arceus-stack.ts`**: the demo stack: KMS-encrypted data lake bucket, Athena results bucket, Glue database, three demo Iceberg tables (`orders`, `events`, `customers`). Repo-only, not in the npm tarball.
|
|
139
|
+
- **`lib/iceberg-evolution-stack.ts`** + **`scripts/integration-test-evolution.sh`**: a parameterized stack and a bash harness that drives four real `cdk deploy`s to prove schema/partition evolution works end-to-end. Repo-only, not in the npm tarball.
|
|
140
140
|
|
|
141
141
|
## Quickstart
|
|
142
142
|
|
|
@@ -160,12 +160,12 @@ Repo source paths (npm consumers import from the package root and get the same e
|
|
|
160
160
|
export PRINCIPAL_ARN="$(aws sts get-caller-identity --query Arn --output text)"
|
|
161
161
|
|
|
162
162
|
npm install
|
|
163
|
-
npx jest #
|
|
163
|
+
npx jest # runs the suite; coverage floor in jest.config.js
|
|
164
164
|
npx cdk deploy ArceusStack --require-approval=never
|
|
165
165
|
./scripts/integration-test-evolution.sh # add + rename + drop, via cdk only
|
|
166
166
|
```
|
|
167
167
|
|
|
168
|
-
`cdk ls` will show two stacks
|
|
168
|
+
`cdk ls` will show two stacks: `ArceusStack` (the demo data lake +
|
|
169
169
|
three Iceberg tables) and `IcebergEvolutionStack` (the evolution
|
|
170
170
|
test target driven by `scripts/integration-test-evolution.sh`).
|
|
171
171
|
Deploy only `ArceusStack` for the quickstart; the evolution stack
|
|
@@ -215,8 +215,8 @@ new IcebergTable(this, 'Users', {
|
|
|
215
215
|
});
|
|
216
216
|
```
|
|
217
217
|
|
|
218
|
-
A table that exercises most of the surface
|
|
219
|
-
nested types, identifier fields, table properties, removal policy.
|
|
218
|
+
A table that exercises most of the surface (partitions, sort order,
|
|
219
|
+
nested types, identifier fields, table properties, removal policy).
|
|
220
220
|
This is the exact shape `ArceusStack` uses for the `orders` demo
|
|
221
221
|
table, so the column list / partition spec / properties round-trip
|
|
222
222
|
straight to the live metadata.json below.
|
|
@@ -488,7 +488,7 @@ underlying Iceberg `metadata.json` after each:
|
|
|
488
488
|
| 3 | **RENAME** `email` → `contact_email` (id 2 preserved), **ADD** partition `bucket(8)(customer_id)` | rename | + `bucket(8)(customer_id)` |
|
|
489
489
|
| 4 | **DROP** column `region` (id 4 stays retired), **DROP** partition `bucket(8)(customer_id)` | − `region` | − `customer_id_bucket` |
|
|
490
490
|
|
|
491
|
-
Last script run output (abridged
|
|
491
|
+
Last script run output (abridged: `cdk deploy` chatter and the
|
|
492
492
|
per-Athena-query state polling lines are omitted; the assertion
|
|
493
493
|
output is verbatim):
|
|
494
494
|
|
|
@@ -583,16 +583,17 @@ under `storageDescriptor`.
|
|
|
583
583
|
Even setting just `tableInput: { name: 'foo' }` next to
|
|
584
584
|
`openTableFormatInput` returns
|
|
585
585
|
`"Table metadata is expected only via TableInput or via IcebergTableInputProperties inside OpenTableFormatInput"`.
|
|
586
|
-
The construct never emits `tableInput
|
|
586
|
+
The construct never emits `tableInput`; the table-level comment goes
|
|
587
587
|
into `tableProperties['comment']`, which lives inside
|
|
588
588
|
`icebergTableInput.properties`.
|
|
589
589
|
|
|
590
|
-
(There is a third footgun
|
|
590
|
+
(There is a third footgun, field-id reuse after a column drop, that
|
|
591
591
|
the construct does **not** prevent. See the next section.)
|
|
592
592
|
|
|
593
593
|
## Known limitations
|
|
594
594
|
|
|
595
595
|
- **Field-id reuse is not detected across deploys.** If you drop a column with `id = 5` and then add a different column with `id = 5` in a later deploy, Glue accepts the UPDATE and Iceberg's metadata silently violates the "never reuse a retired id" invariant. Readers projecting old snapshots will surface deleted data under the new field's name. The construct enforces uniqueness **within one deploy** (`duplicate column id N` validator), but it doesn't compare against the live table state. The safe workflow is to always pin `id` explicitly and treat dropped ids as retired forever; never let CDK reassign an id that has ever been used.
|
|
596
|
+
- **Partition field ids are positional and not pinnable.** The construct allocates partition `fieldId` densely from 1000 in the order partitions appear in `partitionSpec`. Reordering the array across deploys reassigns those ids for unchanged logical partitions, which is the partition-spec analog of the column-id-reuse footgun above. There is no `IcebergPartitionField.fieldId` pinning prop today. The safe workflow is append-only: add new partition fields at the end of `partitionSpec`, and only drop the trailing ones.
|
|
596
597
|
- **CREATE-only metadata operation.** The CFN `IcebergInput.metadataOperation` only accepts `CREATE`; the construct always emits that. Subsequent deploys use Glue's normal `UpdateTable` path, which writes new Iceberg metadata in-place.
|
|
597
598
|
- **Format version is immutable after CREATE.** The `formatVersion` prop is read once at table creation; changing it later requires a destroy + recreate.
|
|
598
599
|
- **`merge-on-read` requires v2.** The construct rejects `write.{delete,update,merge}.mode = merge-on-read` on a v1 table at synth time.
|
|
@@ -608,7 +609,7 @@ npx jest # runs every suite under test/, prints coverage at the end
|
|
|
608
609
|
Coverage is gated at 95% statements / 95% branches / 95% lines / 95% functions
|
|
609
610
|
on `lib/**/*.ts` via the `coverageThreshold.global` block in `jest.config.js`.
|
|
610
611
|
A failing gate fails the suite, so the README does not paste a transcript that
|
|
611
|
-
would drift after the next refactor
|
|
612
|
+
would drift after the next refactor; run the command locally for the live
|
|
612
613
|
numbers.
|
|
613
614
|
|
|
614
615
|
## Project layout
|
|
@@ -657,5 +658,5 @@ arceus/
|
|
|
657
658
|
├── tsconfig.json # Wide include (used by ESLint and dev)
|
|
658
659
|
├── tsconfig.build.json # Narrow include (used by `npm publish`)
|
|
659
660
|
├── jest.config.js
|
|
660
|
-
└── eslint.config.
|
|
661
|
+
└── eslint.config.js
|
|
661
662
|
```
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
import { Grant, IGrantable } from 'aws-cdk-lib/aws-iam';
|
|
2
|
+
export declare const READ_TABLE_ACTIONS: string[];
|
|
3
|
+
export declare const WRITE_TABLE_ACTIONS: string[];
|
|
4
|
+
export declare const READ_S3_LIST_ACTIONS: string[];
|
|
5
|
+
export declare const READ_S3_BUCKET_ACTIONS: string[];
|
|
6
|
+
export declare const WRITE_S3_BUCKET_ACTIONS: string[];
|
|
7
|
+
export declare const READ_S3_OBJECT_ACTIONS: string[];
|
|
8
|
+
export declare const WRITE_S3_OBJECT_ACTIONS: string[];
|
|
9
|
+
/**
|
|
10
|
+
* Issue the four policy statements that scope an Iceberg-table grant
|
|
11
|
+
* correctly: Glue actions on the table ARN, S3 list-bucket actions
|
|
12
|
+
* on the bucket ARN with an `s3:prefix` condition so only the
|
|
13
|
+
* table's own prefix can be listed, S3 bucket-level actions that DO
|
|
14
|
+
* NOT support the `s3:prefix` condition (e.g. `GetBucketLocation`,
|
|
15
|
+
* `ListBucketMultipartUploads`) on the bucket ARN with no condition
|
|
16
|
+
* — including them in the conditioned statement would silently deny
|
|
17
|
+
* them at runtime — and S3 object-level actions on the
|
|
18
|
+
* `bucket/prefix*` ARN. Returns the table-actions grant (any of the
|
|
19
|
+
* four is sufficient for the `Grant` API contract; the rest attach
|
|
20
|
+
* as side effects).
|
|
21
|
+
*
|
|
22
|
+
* @internal
|
|
23
|
+
*/
|
|
24
|
+
export declare function grantSplit(grantee: IGrantable, args: {
|
|
25
|
+
tableArn: string;
|
|
26
|
+
bucketArn: string;
|
|
27
|
+
objectArn: string;
|
|
28
|
+
prefixGlob: string;
|
|
29
|
+
tableActions: string[];
|
|
30
|
+
listActions: string[];
|
|
31
|
+
bucketActions: string[];
|
|
32
|
+
objectActions: string[];
|
|
33
|
+
}): Grant;
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.WRITE_S3_OBJECT_ACTIONS = exports.READ_S3_OBJECT_ACTIONS = exports.WRITE_S3_BUCKET_ACTIONS = exports.READ_S3_BUCKET_ACTIONS = exports.READ_S3_LIST_ACTIONS = exports.WRITE_TABLE_ACTIONS = exports.READ_TABLE_ACTIONS = void 0;
|
|
4
|
+
exports.grantSplit = grantSplit;
|
|
5
|
+
const aws_iam_1 = require("aws-cdk-lib/aws-iam");
|
|
6
|
+
/// /////////////////////////////////////////////////
|
|
7
|
+
/// /////////////////////////////////////////////////
|
|
8
|
+
// Grant action sets
|
|
9
|
+
exports.READ_TABLE_ACTIONS = [
|
|
10
|
+
'glue:BatchGetPartition',
|
|
11
|
+
'glue:GetPartition',
|
|
12
|
+
'glue:GetPartitions',
|
|
13
|
+
'glue:GetTable',
|
|
14
|
+
'glue:GetTables',
|
|
15
|
+
'glue:GetTableVersion',
|
|
16
|
+
'glue:GetTableVersions',
|
|
17
|
+
];
|
|
18
|
+
exports.WRITE_TABLE_ACTIONS = [
|
|
19
|
+
'glue:BatchCreatePartition',
|
|
20
|
+
'glue:BatchDeletePartition',
|
|
21
|
+
'glue:CreatePartition',
|
|
22
|
+
'glue:DeletePartition',
|
|
23
|
+
'glue:UpdatePartition',
|
|
24
|
+
'glue:UpdateTable',
|
|
25
|
+
];
|
|
26
|
+
/// S3 actions that operate at the bucket level AND support the
|
|
27
|
+
/// `s3:prefix` request condition key (per the S3 docs). Granted on
|
|
28
|
+
/// the bucket ARN with a `StringLike s3:prefix = [<prefix>*, <prefix>]`
|
|
29
|
+
/// condition so the grantee can only list the table's own prefix.
|
|
30
|
+
exports.READ_S3_LIST_ACTIONS = [
|
|
31
|
+
's3:ListBucket',
|
|
32
|
+
];
|
|
33
|
+
/// S3 actions that operate at the bucket level but DO NOT support
|
|
34
|
+
/// `s3:prefix`. Granted on the bucket ARN with no condition — adding
|
|
35
|
+
/// one would silently deny these actions at runtime even though they
|
|
36
|
+
/// appear in the policy document.
|
|
37
|
+
exports.READ_S3_BUCKET_ACTIONS = [
|
|
38
|
+
's3:GetBucketLocation',
|
|
39
|
+
];
|
|
40
|
+
exports.WRITE_S3_BUCKET_ACTIONS = [
|
|
41
|
+
's3:ListBucketMultipartUploads',
|
|
42
|
+
];
|
|
43
|
+
/// S3 actions that operate at the object level. Must be granted on
|
|
44
|
+
/// the `bucket/prefix*` ARN.
|
|
45
|
+
exports.READ_S3_OBJECT_ACTIONS = [
|
|
46
|
+
's3:GetObject',
|
|
47
|
+
];
|
|
48
|
+
exports.WRITE_S3_OBJECT_ACTIONS = [
|
|
49
|
+
's3:PutObject',
|
|
50
|
+
's3:DeleteObject',
|
|
51
|
+
's3:AbortMultipartUpload',
|
|
52
|
+
's3:ListMultipartUploadParts',
|
|
53
|
+
];
|
|
54
|
+
/**
|
|
55
|
+
* Issue the four policy statements that scope an Iceberg-table grant
|
|
56
|
+
* correctly: Glue actions on the table ARN, S3 list-bucket actions
|
|
57
|
+
* on the bucket ARN with an `s3:prefix` condition so only the
|
|
58
|
+
* table's own prefix can be listed, S3 bucket-level actions that DO
|
|
59
|
+
* NOT support the `s3:prefix` condition (e.g. `GetBucketLocation`,
|
|
60
|
+
* `ListBucketMultipartUploads`) on the bucket ARN with no condition
|
|
61
|
+
* — including them in the conditioned statement would silently deny
|
|
62
|
+
* them at runtime — and S3 object-level actions on the
|
|
63
|
+
* `bucket/prefix*` ARN. Returns the table-actions grant (any of the
|
|
64
|
+
* four is sufficient for the `Grant` API contract; the rest attach
|
|
65
|
+
* as side effects).
|
|
66
|
+
*
|
|
67
|
+
* @internal
|
|
68
|
+
*/
|
|
69
|
+
function grantSplit(grantee, args) {
|
|
70
|
+
const tableGrant = aws_iam_1.Grant.addToPrincipal({
|
|
71
|
+
grantee,
|
|
72
|
+
actions: args.tableActions,
|
|
73
|
+
resourceArns: [
|
|
74
|
+
args.tableArn,
|
|
75
|
+
],
|
|
76
|
+
});
|
|
77
|
+
if (args.listActions.length > 0) {
|
|
78
|
+
aws_iam_1.Grant.addToPrincipal({
|
|
79
|
+
grantee,
|
|
80
|
+
actions: args.listActions,
|
|
81
|
+
resourceArns: [
|
|
82
|
+
args.bucketArn,
|
|
83
|
+
],
|
|
84
|
+
conditions: {
|
|
85
|
+
StringLike: {
|
|
86
|
+
's3:prefix': [
|
|
87
|
+
args.prefixGlob,
|
|
88
|
+
args.prefixGlob.replace(/\*$/, ''),
|
|
89
|
+
],
|
|
90
|
+
},
|
|
91
|
+
},
|
|
92
|
+
});
|
|
93
|
+
}
|
|
94
|
+
if (args.bucketActions.length > 0) {
|
|
95
|
+
aws_iam_1.Grant.addToPrincipal({
|
|
96
|
+
grantee,
|
|
97
|
+
actions: args.bucketActions,
|
|
98
|
+
resourceArns: [
|
|
99
|
+
args.bucketArn,
|
|
100
|
+
],
|
|
101
|
+
});
|
|
102
|
+
}
|
|
103
|
+
aws_iam_1.Grant.addToPrincipal({
|
|
104
|
+
grantee,
|
|
105
|
+
actions: args.objectActions,
|
|
106
|
+
resourceArns: [
|
|
107
|
+
args.objectArn,
|
|
108
|
+
],
|
|
109
|
+
});
|
|
110
|
+
return tableGrant;
|
|
111
|
+
}
|
|
112
|
+
//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiaWNlYmVyZy10YWJsZS1ncmFudHMuanMiLCJzb3VyY2VSb290IjoiIiwic291cmNlcyI6WyIuLi8uLi8uLi9saWIvaWNlYmVyZy9pY2ViZXJnLXRhYmxlLWdyYW50cy50cyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOzs7QUE0RUEsZ0NBc0RDO0FBbElELGlEQUc2QjtBQUU3QixxREFBcUQ7QUFDckQscURBQXFEO0FBQ3JELG9CQUFvQjtBQUVQLFFBQUEsa0JBQWtCLEdBQUc7SUFDOUIsd0JBQXdCO0lBQ3hCLG1CQUFtQjtJQUNuQixvQkFBb0I7SUFDcEIsZUFBZTtJQUNmLGdCQUFnQjtJQUNoQixzQkFBc0I7SUFDdEIsdUJBQXVCO0NBQzFCLENBQUM7QUFFVyxRQUFBLG1CQUFtQixHQUFHO0lBQy9CLDJCQUEyQjtJQUMzQiwyQkFBMkI7SUFDM0Isc0JBQXNCO0lBQ3RCLHNCQUFzQjtJQUN0QixzQkFBc0I7SUFDdEIsa0JBQWtCO0NBQ3JCLENBQUM7QUFFRiwrREFBK0Q7QUFDL0QsbUVBQW1FO0FBQ25FLHdFQUF3RTtBQUN4RSxrRUFBa0U7QUFDckQsUUFBQSxvQkFBb0IsR0FBRztJQUNoQyxlQUFlO0NBQ2xCLENBQUM7QUFFRixrRUFBa0U7QUFDbEUscUVBQXFFO0FBQ3JFLHFFQUFxRTtBQUNyRSxrQ0FBa0M7QUFDckIsUUFBQSxzQkFBc0IsR0FBRztJQUNsQyxzQkFBc0I7Q0FDekIsQ0FBQztBQUVXLFFBQUEsdUJBQXVCLEdBQUc7SUFDbkMsK0JBQStCO0NBQ2xDLENBQUM7QUFFRixtRUFBbUU7QUFDbkUsNkJBQTZCO0FBQ2hCLFFBQUEsc0JBQXNCLEdBQUc7SUFDbEMsY0FBYztDQUNqQixDQUFDO0FBRVcsUUFBQSx1QkFBdUIsR0FBRztJQUNuQyxjQUFjO0lBQ2QsaUJBQWlCO0lBQ2pCLHlCQUF5QjtJQUN6Qiw2QkFBNkI7Q0FDaEMsQ0FBQztBQUVGOzs7Ozs7Ozs7Ozs7OztHQWNHO0FBQ0gsU0FBZ0IsVUFBVSxDQUN0QixPQUFtQixFQUNuQixJQVNDO0lBRUQsTUFBTSxVQUFVLEdBQUcsZUFBSyxDQUFDLGNBQWMsQ0FBQztRQUNwQyxPQUFPO1FBQ1AsT0FBTyxFQUFFLElBQUksQ0FBQyxZQUFZO1FBQzFCLFlBQVksRUFBRTtZQUNWLElBQUksQ0FBQyxRQUFRO1NBQ2hCO0tBQ0osQ0FBQyxDQUFDO0lBQ0gsSUFBSSxJQUFJLENBQUMsV0FBVyxDQUFDLE1BQU0sR0FBRyxDQUFDLEVBQUUsQ0FBQztRQUM5QixlQUFLLENBQUMsY0FBYyxDQUFDO1lBQ2pCLE9BQU87WUFDUCxPQUFPLEVBQUUsSUFBSSxDQUFDLFdBQVc7WUFDekIsWUFBWSxFQUFFO2dCQUNWLElBQUksQ0FBQyxTQUFTO2FBQ2pCO1lBQ0QsVUFBVSxFQUFFO2dCQUNSLFVBQVUsRUFBRTtvQkFDUixXQUFXLEVBQUU7d0JBQ1QsSUFBSSxDQUFDLFVBQVU7d0JBQ2YsSUFBSSxDQUFDLFVBQVUsQ0FBQyxPQUFPLENBQUMsS0FBSyxFQUFFLEVBQUUsQ0FBQztxQkFDckM7aUJBQ0o7YUFDSjtTQUNKLENBQUMsQ0FBQztJQUNQLENBQUM7SUFDRCxJQUFJLElBQUksQ0FBQyxhQUFhLENBQUMsTUFBTSxHQUFHLENBQUMsRUFBRSxDQUFDO1FBQ2hDLGVBQUssQ0FBQyxjQUFjLENBQUM7WUFDakIsT0FBTztZQUNQLE9BQU8sRUFBRSxJQUFJLENBQUMsYUFBYTtZQUMzQixZQUFZLEVBQUU7Z0JBQ1YsSUFBSSxDQUFDLFNBQVM7YUFDakI7U0FDSixDQUFDLENBQUM7SUFDUCxDQUFDO0lBQ0QsZUFBSyxDQUFDLGNBQWMsQ0FBQztRQUNqQixPQUFPO1FBQ1AsT0FBTyxFQUFFLElBQUksQ0FBQyxhQUFhO1FBQzNCLFlBQVksRUFBRTtZQUNWLElBQUksQ0FBQyxTQUFTO1NBQ2pCO0tBQ0osQ0FBQyxDQUFDO0lBQ0gsT0FBTyxVQUFVLENBQUM7QUFDdEIsQ0FBQyIsInNvdXJjZXNDb250ZW50IjpbImltcG9ydCB7XG4gICAgR3JhbnQsXG4gICAgSUdyYW50YWJsZSxcbn0gZnJvbSAnYXdzLWNkay1saWIvYXdzLWlhbSc7XG5cbi8vLyAvLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vXG4vLy8gLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vL1xuLy8gR3JhbnQgYWN0aW9uIHNldHNcblxuZXhwb3J0IGNvbnN0IFJFQURfVEFCTEVfQUNUSU9OUyA9IFtcbiAgICAnZ2x1ZTpCYXRjaEdldFBhcnRpdGlvbicsXG4gICAgJ2dsdWU6R2V0UGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpHZXRQYXJ0aXRpb25zJyxcbiAgICAnZ2x1ZTpHZXRUYWJsZScsXG4gICAgJ2dsdWU6R2V0VGFibGVzJyxcbiAgICAnZ2x1ZTpHZXRUYWJsZVZlcnNpb24nLFxuICAgICdnbHVlOkdldFRhYmxlVmVyc2lvbnMnLFxuXTtcblxuZXhwb3J0IGNvbnN0IFdSSVRFX1RBQkxFX0FDVElPTlMgPSBbXG4gICAgJ2dsdWU6QmF0Y2hDcmVhdGVQYXJ0aXRpb24nLFxuICAgICdnbHVlOkJhdGNoRGVsZXRlUGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpDcmVhdGVQYXJ0aXRpb24nLFxuICAgICdnbHVlOkRlbGV0ZVBhcnRpdGlvbicsXG4gICAgJ2dsdWU6VXBkYXRlUGFydGl0aW9uJyxcbiAgICAnZ2x1ZTpVcGRhdGVUYWJsZScsXG5dO1xuXG4vLy8gUzMgYWN0aW9ucyB0aGF0IG9wZXJhdGUgYXQgdGhlIGJ1Y2tldCBsZXZlbCBBTkQgc3VwcG9ydCB0aGVcbi8vLyBgczM6cHJlZml4YCByZXF1ZXN0IGNvbmRpdGlvbiBrZXkgKHBlciB0aGUgUzMgZG9jcykuIEdyYW50ZWQgb25cbi8vLyB0aGUgYnVja2V0IEFSTiB3aXRoIGEgYFN0cmluZ0xpa2UgczM6cHJlZml4ID0gWzxwcmVmaXg+KiwgPHByZWZpeD5dYFxuLy8vIGNvbmRpdGlvbiBzbyB0aGUgZ3JhbnRlZSBjYW4gb25seSBsaXN0IHRoZSB0YWJsZSdzIG93biBwcmVmaXguXG5leHBvcnQgY29uc3QgUkVBRF9TM19MSVNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOkxpc3RCdWNrZXQnLFxuXTtcblxuLy8vIFMzIGFjdGlvbnMgdGhhdCBvcGVyYXRlIGF0IHRoZSBidWNrZXQgbGV2ZWwgYnV0IERPIE5PVCBzdXBwb3J0XG4vLy8gYHMzOnByZWZpeGAuIEdyYW50ZWQgb24gdGhlIGJ1Y2tldCBBUk4gd2l0aCBubyBjb25kaXRpb24g4oCUIGFkZGluZ1xuLy8vIG9uZSB3b3VsZCBzaWxlbnRseSBkZW55IHRoZXNlIGFjdGlvbnMgYXQgcnVudGltZSBldmVuIHRob3VnaCB0aGV5XG4vLy8gYXBwZWFyIGluIHRoZSBwb2xpY3kgZG9jdW1lbnQuXG5leHBvcnQgY29uc3QgUkVBRF9TM19CVUNLRVRfQUNUSU9OUyA9IFtcbiAgICAnczM6R2V0QnVja2V0TG9jYXRpb24nLFxuXTtcblxuZXhwb3J0IGNvbnN0IFdSSVRFX1MzX0JVQ0tFVF9BQ1RJT05TID0gW1xuICAgICdzMzpMaXN0QnVja2V0TXVsdGlwYXJ0VXBsb2FkcycsXG5dO1xuXG4vLy8gUzMgYWN0aW9ucyB0aGF0IG9wZXJhdGUgYXQgdGhlIG9iamVjdCBsZXZlbC4gTXVzdCBiZSBncmFudGVkIG9uXG4vLy8gdGhlIGBidWNrZXQvcHJlZml4KmAgQVJOLlxuZXhwb3J0IGNvbnN0IFJFQURfUzNfT0JKRUNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOkdldE9iamVjdCcsXG5dO1xuXG5leHBvcnQgY29uc3QgV1JJVEVfUzNfT0JKRUNUX0FDVElPTlMgPSBbXG4gICAgJ3MzOlB1dE9iamVjdCcsXG4gICAgJ3MzOkRlbGV0ZU9iamVjdCcsXG4gICAgJ3MzOkFib3J0TXVsdGlwYXJ0VXBsb2FkJyxcbiAgICAnczM6TGlzdE11bHRpcGFydFVwbG9hZFBhcnRzJyxcbl07XG5cbi8qKlxuICogSXNzdWUgdGhlIGZvdXIgcG9saWN5IHN0YXRlbWVudHMgdGhhdCBzY29wZSBhbiBJY2ViZXJnLXRhYmxlIGdyYW50XG4gKiBjb3JyZWN0bHk6IEdsdWUgYWN0aW9ucyBvbiB0aGUgdGFibGUgQVJOLCBTMyBsaXN0LWJ1Y2tldCBhY3Rpb25zXG4gKiBvbiB0aGUgYnVja2V0IEFSTiB3aXRoIGFuIGBzMzpwcmVmaXhgIGNvbmRpdGlvbiBzbyBvbmx5IHRoZVxuICogdGFibGUncyBvd24gcHJlZml4IGNhbiBiZSBsaXN0ZWQsIFMzIGJ1Y2tldC1sZXZlbCBhY3Rpb25zIHRoYXQgRE9cbiAqIE5PVCBzdXBwb3J0IHRoZSBgczM6cHJlZml4YCBjb25kaXRpb24gKGUuZy4gYEdldEJ1Y2tldExvY2F0aW9uYCxcbiAqIGBMaXN0QnVja2V0TXVsdGlwYXJ0VXBsb2Fkc2ApIG9uIHRoZSBidWNrZXQgQVJOIHdpdGggbm8gY29uZGl0aW9uXG4gKiDigJQgaW5jbHVkaW5nIHRoZW0gaW4gdGhlIGNvbmRpdGlvbmVkIHN0YXRlbWVudCB3b3VsZCBzaWxlbnRseSBkZW55XG4gKiB0aGVtIGF0IHJ1bnRpbWUg4oCUIGFuZCBTMyBvYmplY3QtbGV2ZWwgYWN0aW9ucyBvbiB0aGVcbiAqIGBidWNrZXQvcHJlZml4KmAgQVJOLiBSZXR1cm5zIHRoZSB0YWJsZS1hY3Rpb25zIGdyYW50IChhbnkgb2YgdGhlXG4gKiBmb3VyIGlzIHN1ZmZpY2llbnQgZm9yIHRoZSBgR3JhbnRgIEFQSSBjb250cmFjdDsgdGhlIHJlc3QgYXR0YWNoXG4gKiBhcyBzaWRlIGVmZmVjdHMpLlxuICpcbiAqIEBpbnRlcm5hbFxuICovXG5leHBvcnQgZnVuY3Rpb24gZ3JhbnRTcGxpdChcbiAgICBncmFudGVlOiBJR3JhbnRhYmxlLFxuICAgIGFyZ3M6IHtcbiAgICAgICAgdGFibGVBcm46IHN0cmluZztcbiAgICAgICAgYnVja2V0QXJuOiBzdHJpbmc7XG4gICAgICAgIG9iamVjdEFybjogc3RyaW5nO1xuICAgICAgICBwcmVmaXhHbG9iOiBzdHJpbmc7XG4gICAgICAgIHRhYmxlQWN0aW9uczogc3RyaW5nW107XG4gICAgICAgIGxpc3RBY3Rpb25zOiBzdHJpbmdbXTtcbiAgICAgICAgYnVja2V0QWN0aW9uczogc3RyaW5nW107XG4gICAgICAgIG9iamVjdEFjdGlvbnM6IHN0cmluZ1tdO1xuICAgIH0sXG4pOiBHcmFudCB7XG4gICAgY29uc3QgdGFibGVHcmFudCA9IEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgZ3JhbnRlZSxcbiAgICAgICAgYWN0aW9uczogYXJncy50YWJsZUFjdGlvbnMsXG4gICAgICAgIHJlc291cmNlQXJuczogW1xuICAgICAgICAgICAgYXJncy50YWJsZUFybixcbiAgICAgICAgXSxcbiAgICB9KTtcbiAgICBpZiAoYXJncy5saXN0QWN0aW9ucy5sZW5ndGggPiAwKSB7XG4gICAgICAgIEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgICAgIGdyYW50ZWUsXG4gICAgICAgICAgICBhY3Rpb25zOiBhcmdzLmxpc3RBY3Rpb25zLFxuICAgICAgICAgICAgcmVzb3VyY2VBcm5zOiBbXG4gICAgICAgICAgICAgICAgYXJncy5idWNrZXRBcm4sXG4gICAgICAgICAgICBdLFxuICAgICAgICAgICAgY29uZGl0aW9uczoge1xuICAgICAgICAgICAgICAgIFN0cmluZ0xpa2U6IHtcbiAgICAgICAgICAgICAgICAgICAgJ3MzOnByZWZpeCc6IFtcbiAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3MucHJlZml4R2xvYixcbiAgICAgICAgICAgICAgICAgICAgICAgIGFyZ3MucHJlZml4R2xvYi5yZXBsYWNlKC9cXCokLywgJycpLFxuICAgICAgICAgICAgICAgICAgICBdLFxuICAgICAgICAgICAgICAgIH0sXG4gICAgICAgICAgICB9LFxuICAgICAgICB9KTtcbiAgICB9XG4gICAgaWYgKGFyZ3MuYnVja2V0QWN0aW9ucy5sZW5ndGggPiAwKSB7XG4gICAgICAgIEdyYW50LmFkZFRvUHJpbmNpcGFsKHtcbiAgICAgICAgICAgIGdyYW50ZWUsXG4gICAgICAgICAgICBhY3Rpb25zOiBhcmdzLmJ1Y2tldEFjdGlvbnMsXG4gICAgICAgICAgICByZXNvdXJjZUFybnM6IFtcbiAgICAgICAgICAgICAgICBhcmdzLmJ1Y2tldEFybixcbiAgICAgICAgICAgIF0sXG4gICAgICAgIH0pO1xuICAgIH1cbiAgICBHcmFudC5hZGRUb1ByaW5jaXBhbCh7XG4gICAgICAgIGdyYW50ZWUsXG4gICAgICAgIGFjdGlvbnM6IGFyZ3Mub2JqZWN0QWN0aW9ucyxcbiAgICAgICAgcmVzb3VyY2VBcm5zOiBbXG4gICAgICAgICAgICBhcmdzLm9iamVjdEFybixcbiAgICAgICAgXSxcbiAgICB9KTtcbiAgICByZXR1cm4gdGFibGVHcmFudDtcbn1cbiJdfQ==
|
|
@@ -0,0 +1,41 @@
|
|
|
1
|
+
import { CfnTable } from 'aws-cdk-lib/aws-glue';
|
|
2
|
+
import { IcebergType } from './iceberg-type';
|
|
3
|
+
import { IcebergDataFormat, IcebergFormatVersion } from './iceberg-table-properties';
|
|
4
|
+
import { IcebergColumn, IcebergPartitionField, IcebergSortField, IcebergTableProps } from './iceberg-table';
|
|
5
|
+
/** @internal */
|
|
6
|
+
export declare function validateProps(props: IcebergTableProps): void;
|
|
7
|
+
/** @internal */
|
|
8
|
+
export declare function normalizeLocation(location: string): string;
|
|
9
|
+
/** @internal */
|
|
10
|
+
export declare function parseS3Uri(uri: string): {
|
|
11
|
+
bucket: string;
|
|
12
|
+
key: string;
|
|
13
|
+
};
|
|
14
|
+
interface RenderedColumn {
|
|
15
|
+
readonly id: number;
|
|
16
|
+
readonly type: IcebergType;
|
|
17
|
+
}
|
|
18
|
+
/** @internal */
|
|
19
|
+
export interface RenderedSchema {
|
|
20
|
+
readonly fields: CfnTable.IcebergStructFieldProperty[];
|
|
21
|
+
readonly columnByName: Map<string, RenderedColumn>;
|
|
22
|
+
}
|
|
23
|
+
/** @internal */
|
|
24
|
+
export declare function renderSchema(columns: IcebergColumn[]): RenderedSchema;
|
|
25
|
+
/** @internal */
|
|
26
|
+
export declare function validatePartitionSpec(partitionSpec: IcebergPartitionField[] | undefined, columnByName: Map<string, RenderedColumn>): void;
|
|
27
|
+
/** @internal */
|
|
28
|
+
export declare function validateSortOrder(sortOrder: IcebergSortField[] | undefined, columnByName: Map<string, RenderedColumn>): void;
|
|
29
|
+
/** @internal */
|
|
30
|
+
export declare function renderPartitionSpec(partitionSpec: IcebergPartitionField[] | undefined, columnByName: Map<string, RenderedColumn>): CfnTable.IcebergPartitionSpecProperty | undefined;
|
|
31
|
+
/** @internal */
|
|
32
|
+
export declare function renderSortOrder(sortOrder: IcebergSortField[] | undefined, columnByName: Map<string, RenderedColumn>): CfnTable.IcebergSortOrderProperty | undefined;
|
|
33
|
+
/** @internal */
|
|
34
|
+
export declare function resolveIdentifierFieldIds(identifierFieldNames: string[] | undefined, columnByName: Map<string, RenderedColumn>): number[] | undefined;
|
|
35
|
+
/** @internal */
|
|
36
|
+
export declare function mergeProperties(dataFormat: IcebergDataFormat, formatVersion: IcebergFormatVersion, user: {
|
|
37
|
+
[key: string]: string;
|
|
38
|
+
} | undefined, comment: string | undefined): {
|
|
39
|
+
[key: string]: string;
|
|
40
|
+
};
|
|
41
|
+
export {};
|
|
@@ -0,0 +1,241 @@
|
|
|
1
|
+
"use strict";
|
|
2
|
+
Object.defineProperty(exports, "__esModule", { value: true });
|
|
3
|
+
exports.validateProps = validateProps;
|
|
4
|
+
exports.normalizeLocation = normalizeLocation;
|
|
5
|
+
exports.parseS3Uri = parseS3Uri;
|
|
6
|
+
exports.renderSchema = renderSchema;
|
|
7
|
+
exports.validatePartitionSpec = validatePartitionSpec;
|
|
8
|
+
exports.validateSortOrder = validateSortOrder;
|
|
9
|
+
exports.renderPartitionSpec = renderPartitionSpec;
|
|
10
|
+
exports.renderSortOrder = renderSortOrder;
|
|
11
|
+
exports.resolveIdentifierFieldIds = resolveIdentifierFieldIds;
|
|
12
|
+
exports.mergeProperties = mergeProperties;
|
|
13
|
+
const iceberg_partition_transform_1 = require("./iceberg-partition-transform");
|
|
14
|
+
const iceberg_table_properties_1 = require("./iceberg-table-properties");
|
|
15
|
+
const iceberg_table_1 = require("./iceberg-table");
|
|
16
|
+
/// /////////////////////////////////////////////////
|
|
17
|
+
/// /////////////////////////////////////////////////
|
|
18
|
+
// Validation
|
|
19
|
+
/** @internal */
|
|
20
|
+
function validateProps(props) {
|
|
21
|
+
if (!props.tableName.match(/^[a-z0-9_]+$/)) {
|
|
22
|
+
throw new Error(`tableName '${props.tableName}' must contain only lowercase letters, digits, and underscores`);
|
|
23
|
+
}
|
|
24
|
+
if (props.columns.length === 0) {
|
|
25
|
+
throw new Error('IcebergTable requires at least one column');
|
|
26
|
+
}
|
|
27
|
+
const names = new Set();
|
|
28
|
+
for (const column of props.columns) {
|
|
29
|
+
if (names.has(column.name)) {
|
|
30
|
+
throw new Error(`duplicate column name: ${column.name}`);
|
|
31
|
+
}
|
|
32
|
+
names.add(column.name);
|
|
33
|
+
}
|
|
34
|
+
if (!props.location.startsWith('s3://')) {
|
|
35
|
+
throw new Error(`location must start with 's3://', got '${props.location}'`);
|
|
36
|
+
}
|
|
37
|
+
}
|
|
38
|
+
/** @internal */
|
|
39
|
+
function normalizeLocation(location) {
|
|
40
|
+
return location.endsWith('/') ? location : `${location}/`;
|
|
41
|
+
}
|
|
42
|
+
/** @internal */
|
|
43
|
+
function parseS3Uri(uri) {
|
|
44
|
+
/// `normalizeLocation` is always called first, so `uri` ends with `/`
|
|
45
|
+
/// and therefore contains at least one `/` after the scheme.
|
|
46
|
+
const rest = uri.substring('s3://'.length);
|
|
47
|
+
const slash = rest.indexOf('/');
|
|
48
|
+
return {
|
|
49
|
+
bucket: rest.substring(0, slash),
|
|
50
|
+
key: rest.substring(slash + 1),
|
|
51
|
+
};
|
|
52
|
+
}
|
|
53
|
+
/** @internal */
|
|
54
|
+
function renderSchema(columns) {
|
|
55
|
+
/// First pass: assign top-level ids. Pinned ids win; the rest are
|
|
56
|
+
/// filled with the smallest unused positive integers. Nested-type
|
|
57
|
+
/// ids continue above the highest top-level id so they never
|
|
58
|
+
/// collide with column ids.
|
|
59
|
+
const topLevelIds = assignTopLevelIds(columns);
|
|
60
|
+
let nextId = Math.max(...topLevelIds, columns.length) + 1;
|
|
61
|
+
const ctx = {
|
|
62
|
+
nextId: () => {
|
|
63
|
+
const id = nextId;
|
|
64
|
+
nextId += 1;
|
|
65
|
+
return id;
|
|
66
|
+
},
|
|
67
|
+
};
|
|
68
|
+
const columnByName = new Map();
|
|
69
|
+
const fields = columns.map((column, index) => {
|
|
70
|
+
const id = topLevelIds[index];
|
|
71
|
+
const typeString = column.type._render(ctx);
|
|
72
|
+
columnByName.set(column.name, {
|
|
73
|
+
id,
|
|
74
|
+
type: column.type,
|
|
75
|
+
});
|
|
76
|
+
return {
|
|
77
|
+
id,
|
|
78
|
+
name: column.name,
|
|
79
|
+
type: typeString,
|
|
80
|
+
required: column.required ?? false,
|
|
81
|
+
doc: column.doc,
|
|
82
|
+
};
|
|
83
|
+
});
|
|
84
|
+
return {
|
|
85
|
+
fields,
|
|
86
|
+
columnByName,
|
|
87
|
+
};
|
|
88
|
+
}
|
|
89
|
+
/**
|
|
90
|
+
* Assign top-level field ids honoring user-pinned ids and filling
|
|
91
|
+
* unspecified ones with the smallest unused positive integer.
|
|
92
|
+
* Throws on duplicate or non-positive ids.
|
|
93
|
+
*/
|
|
94
|
+
function assignTopLevelIds(columns) {
|
|
95
|
+
const taken = new Set();
|
|
96
|
+
for (const column of columns) {
|
|
97
|
+
if (column.id === undefined) {
|
|
98
|
+
continue;
|
|
99
|
+
}
|
|
100
|
+
if (!Number.isInteger(column.id) || column.id < 1) {
|
|
101
|
+
throw new Error(`column '${column.name}' has invalid id ${column.id}; ids must be positive integers`);
|
|
102
|
+
}
|
|
103
|
+
if (taken.has(column.id)) {
|
|
104
|
+
throw new Error(`duplicate column id ${column.id} on column '${column.name}'`);
|
|
105
|
+
}
|
|
106
|
+
taken.add(column.id);
|
|
107
|
+
}
|
|
108
|
+
let cursor = 1;
|
|
109
|
+
return columns.map((column) => {
|
|
110
|
+
if (column.id !== undefined) {
|
|
111
|
+
return column.id;
|
|
112
|
+
}
|
|
113
|
+
while (taken.has(cursor)) {
|
|
114
|
+
cursor += 1;
|
|
115
|
+
}
|
|
116
|
+
taken.add(cursor);
|
|
117
|
+
const assigned = cursor;
|
|
118
|
+
cursor += 1;
|
|
119
|
+
return assigned;
|
|
120
|
+
});
|
|
121
|
+
}
|
|
122
|
+
/** @internal */
|
|
123
|
+
function validatePartitionSpec(partitionSpec, columnByName) {
|
|
124
|
+
if (partitionSpec === undefined) {
|
|
125
|
+
return;
|
|
126
|
+
}
|
|
127
|
+
const partitionNames = new Set();
|
|
128
|
+
for (const field of partitionSpec) {
|
|
129
|
+
const source = columnByName.get(field.sourceColumn);
|
|
130
|
+
if (source === undefined) {
|
|
131
|
+
throw new Error(`partitionSpec references unknown column '${field.sourceColumn}'`);
|
|
132
|
+
}
|
|
133
|
+
field.transform.validateSourceType(field.sourceColumn, source.type);
|
|
134
|
+
const partitionName = field.name ?? defaultPartitionName(field);
|
|
135
|
+
if (partitionNames.has(partitionName)) {
|
|
136
|
+
throw new Error(`duplicate partition field name: ${partitionName}`);
|
|
137
|
+
}
|
|
138
|
+
partitionNames.add(partitionName);
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
/** @internal */
|
|
142
|
+
function validateSortOrder(sortOrder, columnByName) {
|
|
143
|
+
if (sortOrder === undefined) {
|
|
144
|
+
return;
|
|
145
|
+
}
|
|
146
|
+
if (sortOrder.length === 0) {
|
|
147
|
+
throw new Error('sortOrder must contain at least one field when provided');
|
|
148
|
+
}
|
|
149
|
+
for (const field of sortOrder) {
|
|
150
|
+
const source = columnByName.get(field.sourceColumn);
|
|
151
|
+
if (source === undefined) {
|
|
152
|
+
throw new Error(`sortOrder references unknown column '${field.sourceColumn}'`);
|
|
153
|
+
}
|
|
154
|
+
const transform = field.transform ?? iceberg_partition_transform_1.IcebergPartitionTransform.IDENTITY;
|
|
155
|
+
transform.validateSourceType(field.sourceColumn, source.type);
|
|
156
|
+
}
|
|
157
|
+
}
|
|
158
|
+
/** @internal */
|
|
159
|
+
function renderPartitionSpec(partitionSpec, columnByName) {
|
|
160
|
+
if (partitionSpec === undefined || partitionSpec.length === 0) {
|
|
161
|
+
return undefined;
|
|
162
|
+
}
|
|
163
|
+
const fields = partitionSpec.map((field, index) => ({
|
|
164
|
+
name: field.name ?? defaultPartitionName(field),
|
|
165
|
+
sourceId: columnByName.get(field.sourceColumn).id,
|
|
166
|
+
transform: field.transform.toTransformString(),
|
|
167
|
+
/// Per the Iceberg spec, partition field ids must be in the
|
|
168
|
+
/// range [1000, 9999]; the construct allocates them densely
|
|
169
|
+
/// from 1000 in declaration order. Reordering partitions
|
|
170
|
+
/// across deploys reassigns ids — see the "partition field id
|
|
171
|
+
/// reuse" entry in the README's Known limitations.
|
|
172
|
+
fieldId: 1000 + index,
|
|
173
|
+
}));
|
|
174
|
+
return {
|
|
175
|
+
specId: 0,
|
|
176
|
+
fields,
|
|
177
|
+
};
|
|
178
|
+
}
|
|
179
|
+
/** @internal */
|
|
180
|
+
function renderSortOrder(sortOrder, columnByName) {
|
|
181
|
+
if (sortOrder === undefined || sortOrder.length === 0) {
|
|
182
|
+
return undefined;
|
|
183
|
+
}
|
|
184
|
+
const fields = sortOrder.map((field) => {
|
|
185
|
+
const transform = field.transform ?? iceberg_partition_transform_1.IcebergPartitionTransform.IDENTITY;
|
|
186
|
+
return {
|
|
187
|
+
sourceId: columnByName.get(field.sourceColumn).id,
|
|
188
|
+
transform: transform.toTransformString(),
|
|
189
|
+
direction: field.direction ?? iceberg_table_1.IcebergSortDirection.ASC,
|
|
190
|
+
nullOrder: field.nullOrder ?? iceberg_table_1.IcebergNullOrder.NULLS_LAST,
|
|
191
|
+
};
|
|
192
|
+
});
|
|
193
|
+
return {
|
|
194
|
+
orderId: 1,
|
|
195
|
+
fields,
|
|
196
|
+
};
|
|
197
|
+
}
|
|
198
|
+
function defaultPartitionName(field) {
|
|
199
|
+
const transformString = field.transform.toTransformString();
|
|
200
|
+
if (transformString === 'identity') {
|
|
201
|
+
return field.sourceColumn;
|
|
202
|
+
}
|
|
203
|
+
/// `bucket[16]` -> `sourceColumn_bucket`
|
|
204
|
+
const baseTransform = transformString.replace(/\[.*\]$/, '');
|
|
205
|
+
return `${field.sourceColumn}_${baseTransform}`;
|
|
206
|
+
}
|
|
207
|
+
/** @internal */
|
|
208
|
+
function resolveIdentifierFieldIds(identifierFieldNames, columnByName) {
|
|
209
|
+
if (identifierFieldNames === undefined || identifierFieldNames.length === 0) {
|
|
210
|
+
return undefined;
|
|
211
|
+
}
|
|
212
|
+
const seen = new Set();
|
|
213
|
+
return identifierFieldNames.map((name) => {
|
|
214
|
+
if (seen.has(name)) {
|
|
215
|
+
throw new Error(`duplicate identifier field: ${name}`);
|
|
216
|
+
}
|
|
217
|
+
seen.add(name);
|
|
218
|
+
const column = columnByName.get(name);
|
|
219
|
+
if (column === undefined) {
|
|
220
|
+
throw new Error(`identifierFieldNames references unknown column '${name}'`);
|
|
221
|
+
}
|
|
222
|
+
return column.id;
|
|
223
|
+
});
|
|
224
|
+
}
|
|
225
|
+
/** @internal */
|
|
226
|
+
function mergeProperties(dataFormat, formatVersion, user, comment) {
|
|
227
|
+
if (comment !== undefined && user !== undefined && user.comment !== undefined) {
|
|
228
|
+
throw new Error('comment is set both as the top-level `comment` prop and as `tableProperties.comment`; '
|
|
229
|
+
+ 'set it in exactly one place');
|
|
230
|
+
}
|
|
231
|
+
const merged = {
|
|
232
|
+
[iceberg_table_properties_1.ICEBERG_PROPERTY_KEYS.FORMAT_VERSION]: formatVersion,
|
|
233
|
+
[iceberg_table_properties_1.ICEBERG_PROPERTY_KEYS.WRITE_FORMAT_DEFAULT]: dataFormat,
|
|
234
|
+
...(user ?? {}),
|
|
235
|
+
};
|
|
236
|
+
if (comment !== undefined) {
|
|
237
|
+
merged.comment = comment;
|
|
238
|
+
}
|
|
239
|
+
return merged;
|
|
240
|
+
}
|
|
241
|
+
//# sourceMappingURL=data:application/json;base64,eyJ2ZXJzaW9uIjozLCJmaWxlIjoiaWNlYmVyZy10YWJsZS1yZW5kZXIuanMiLCJzb3VyY2VSb290IjoiIiwic291cmNlcyI6WyIuLi8uLi8uLi9saWIvaWNlYmVyZy9pY2ViZXJnLXRhYmxlLXJlbmRlci50cyJdLCJuYW1lcyI6W10sIm1hcHBpbmdzIjoiOztBQTZCQSxzQ0FtQkM7QUFHRCw4Q0FFQztBQUdELGdDQVNDO0FBa0JELG9DQWtDQztBQXVDRCxzREFzQkM7QUFHRCw4Q0FrQkM7QUFHRCxrREFzQkM7QUFHRCwwQ0FvQkM7QUFhRCw4REFtQkM7QUFHRCwwQ0FxQkM7QUF4U0QsK0VBRXVDO0FBQ3ZDLHlFQUlvQztBQUNwQyxtREFPeUI7QUFFekIscURBQXFEO0FBQ3JELHFEQUFxRDtBQUNyRCxhQUFhO0FBRWIsZ0JBQWdCO0FBQ2hCLFNBQWdCLGFBQWEsQ0FBQyxLQUF3QjtJQUNsRCxJQUFJLENBQUMsS0FBSyxDQUFDLFNBQVMsQ0FBQyxLQUFLLENBQUMsY0FBYyxDQUFDLEVBQUUsQ0FBQztRQUN6QyxNQUFNLElBQUksS0FBSyxDQUNYLGNBQWMsS0FBSyxDQUFDLFNBQVMsZ0VBQWdFLENBQ2hHLENBQUM7SUFDTixDQUFDO0lBQ0QsSUFBSSxLQUFLLENBQUMsT0FBTyxDQUFDLE1BQU0sS0FBSyxDQUFDLEVBQUUsQ0FBQztRQUM3QixNQUFNLElBQUksS0FBSyxDQUFDLDJDQUEyQyxDQUFDLENBQUM7SUFDakUsQ0FBQztJQUNELE1BQU0sS0FBSyxHQUFHLElBQUksR0FBRyxFQUFVLENBQUM7SUFDaEMsS0FBSyxNQUFNLE1BQU0sSUFBSSxLQUFLLENBQUMsT0FBTyxFQUFFLENBQUM7UUFDakMsSUFBSSxLQUFLLENBQUMsR0FBRyxDQUFDLE1BQU0sQ0FBQyxJQUFJLENBQUMsRUFBRSxDQUFDO1lBQ3pCLE1BQU0sSUFBSSxLQUFLLENBQUMsMEJBQTBCLE1BQU0sQ0FBQyxJQUFJLEVBQUUsQ0FBQyxDQUFDO1FBQzdELENBQUM7UUFDRCxLQUFLLENBQUMsR0FBRyxDQUFDLE1BQU0sQ0FBQyxJQUFJLENBQUMsQ0FBQztJQUMzQixDQUFDO0lBQ0QsSUFBSSxDQUFDLEtBQUssQ0FBQyxRQUFRLENBQUMsVUFBVSxDQUFDLE9BQU8sQ0FBQyxFQUFFLENBQUM7UUFDdEMsTUFBTSxJQUFJLEtBQUssQ0FBQywwQ0FBMEMsS0FBSyxDQUFDLFFBQVEsR0FBRyxDQUFDLENBQUM7SUFDakYsQ0FBQztBQUNMLENBQUM7QUFFRCxnQkFBZ0I7QUFDaEIsU0FBZ0IsaUJBQWlCLENBQUMsUUFBZ0I7SUFDOUMsT0FBTyxRQUFRLENBQUMsUUFBUSxDQUFDLEdBQUcsQ0FBQyxDQUFDLENBQUMsQ0FBQyxRQUFRLENBQUMsQ0FBQyxDQUFDLEdBQUcsUUFBUSxHQUFHLENBQUM7QUFDOUQsQ0FBQztBQUVELGdCQUFnQjtBQUNoQixTQUFnQixVQUFVLENBQUMsR0FBVztJQUNsQyxzRUFBc0U7SUFDdEUsNkRBQTZEO0lBQzdELE1BQU0sSUFBSSxHQUFHLEdBQUcsQ0FBQyxTQUFTLENBQUMsT0FBTyxDQUFDLE1BQU0sQ0FBQyxDQUFDO0lBQzNDLE1BQU0sS0FBSyxHQUFHLElBQUksQ0FBQyxPQUFPLENBQUMsR0FBRyxDQUFDLENBQUM7SUFDaEMsT0FBTztRQUNILE1BQU0sRUFBRSxJQUFJLENBQUMsU0FBUyxDQUFDLENBQUMsRUFBRSxLQUFLLENBQUM7UUFDaEMsR0FBRyxFQUFFLElBQUksQ0FBQyxTQUFTLENBQUMsS0FBSyxHQUFHLENBQUMsQ0FBQztLQUNqQyxDQUFDO0FBQ04sQ0FBQztBQWlCRCxnQkFBZ0I7QUFDaEIsU0FBZ0IsWUFBWSxDQUFDLE9BQXdCO0lBQ2pELGtFQUFrRTtJQUNsRSxrRUFBa0U7SUFDbEUsNkRBQTZEO0lBQzdELDRCQUE0QjtJQUM1QixNQUFNLFdBQVcsR0FBRyxpQkFBaUIsQ0FBQyxPQUFPLENBQUMsQ0FBQztJQUMvQyxJQUFJLE1BQU0sR0FBRyxJQUFJLENBQUMsR0FBRyxDQUFDLEdBQUcsV0FBVyxFQUFFLE9BQU8sQ0FBQyxNQUFNLENBQUMsR0FBRyxDQUFDLENBQUM7SUFDMUQsTUFBTSxHQUFHLEdBQXlCO1FBQzlCLE1BQU0sRUFBRSxHQUFHLEVBQUU7WUFDVCxNQUFNLEVBQUUsR0FBRyxNQUFNLENBQUM7WUFDbEIsTUFBTSxJQUFJLENBQUMsQ0FBQztZQUNaLE9BQU8sRUFBRSxDQUFDO1FBQ2QsQ0FBQztLQUNKLENBQUM7SUFDRixNQUFNLFlBQVksR0FBRyxJQUFJLEdBQUcsRUFBMEIsQ0FBQztJQUN2RCxNQUFNLE1BQU0sR0FBMEMsT0FBTyxDQUFDLEdBQUcsQ0FBQyxDQUFDLE1BQU0sRUFBRSxLQUFLLEVBQUUsRUFBRTtRQUNoRixNQUFNLEVBQUUsR0FBRyxXQUFXLENBQUMsS0FBSyxDQUFDLENBQUM7UUFDOUIsTUFBTSxVQUFVLEdBQUcsTUFBTSxDQUFDLElBQUksQ0FBQyxPQUFPLENBQUMsR0FBRyxDQUFDLENBQUM7UUFDNUMsWUFBWSxDQUFDLEdBQUcsQ0FBQyxNQUFNLENBQUMsSUFBSSxFQUFFO1lBQzFCLEVBQUU7WUFDRixJQUFJLEVBQUUsTUFBTSxDQUFDLElBQUk7U0FDcEIsQ0FBQyxDQUFDO1FBQ0gsT0FBTztZQUNILEVBQUU7WUFDRixJQUFJLEVBQUUsTUFBTSxDQUFDLElBQUk7WUFDakIsSUFBSSxFQUFFLFVBQVU7WUFDaEIsUUFBUSxFQUFFLE1BQU0sQ0FBQyxRQUFRLElBQUksS0FBSztZQUNsQyxHQUFHLEVBQUUsTUFBTSxDQUFDLEdBQUc7U0FDbEIsQ0FBQztJQUNOLENBQUMsQ0FBQyxDQUFDO0lBQ0gsT0FBTztRQUNILE1BQU07UUFDTixZQUFZO0tBQ2YsQ0FBQztBQUNOLENBQUM7QUFFRDs7OztHQUlHO0FBQ0gsU0FBUyxpQkFBaUIsQ0FBQyxPQUF3QjtJQUMvQyxNQUFNLEtBQUssR0FBRyxJQUFJLEdBQUcsRUFBVSxDQUFDO0lBQ2hDLEtBQUssTUFBTSxNQUFNLElBQUksT0FBTyxFQUFFLENBQUM7UUFDM0IsSUFBSSxNQUFNLENBQUMsRUFBRSxLQUFLLFNBQVMsRUFBRSxDQUFDO1lBQzFCLFNBQVM7UUFDYixDQUFDO1FBQ0QsSUFBSSxDQUFDLE1BQU0sQ0FBQyxTQUFTLENBQUMsTUFBTSxDQUFDLEVBQUUsQ0FBQyxJQUFJLE1BQU0sQ0FBQyxFQUFFLEdBQUcsQ0FBQyxFQUFFLENBQUM7WUFDaEQsTUFBTSxJQUFJLEtBQUssQ0FDWCxXQUFXLE1BQU0sQ0FBQyxJQUFJLG9CQUFvQixNQUFNLENBQUMsRUFBRSxpQ0FBaUMsQ0FDdkYsQ0FBQztRQUNOLENBQUM7UUFDRCxJQUFJLEtBQUssQ0FBQyxHQUFHLENBQUMsTUFBTSxDQUFDLEVBQUUsQ0FBQyxFQUFFLENBQUM7WUFDdkIsTUFBTSxJQUFJLEtBQUssQ0FBQyx1QkFBdUIsTUFBTSxDQUFDLEVBQUUsZUFBZSxNQUFNLENBQUMsSUFBSSxHQUFHLENBQUMsQ0FBQztRQUNuRixDQUFDO1FBQ0QsS0FBSyxDQUFDLEdBQUcsQ0FBQyxNQUFNLENBQUMsRUFBRSxDQUFDLENBQUM7SUFDekIsQ0FBQztJQUNELElBQUksTUFBTSxHQUFHLENBQUMsQ0FBQztJQUNmLE9BQU8sT0FBTyxDQUFDLEdBQUcsQ0FBQyxDQUFDLE1BQU0sRUFBRSxFQUFFO1FBQzFCLElBQUksTUFBTSxDQUFDLEVBQUUsS0FBSyxTQUFTLEVBQUUsQ0FBQztZQUMxQixPQUFPLE1BQU0sQ0FBQyxFQUFFLENBQUM7UUFDckIsQ0FBQztRQUNELE9BQU8sS0FBSyxDQUFDLEdBQUcsQ0FBQyxNQUFNLENBQUMsRUFBRSxDQUFDO1lBQ3ZCLE1BQU0sSUFBSSxDQUFDLENBQUM7UUFDaEIsQ0FBQztRQUNELEtBQUssQ0FBQyxHQUFHLENBQUMsTUFBTSxDQUFDLENBQUM7UUFDbEIsTUFBTSxRQUFRLEdBQUcsTUFBTSxDQUFDO1FBQ3hCLE1BQU0sSUFBSSxDQUFDLENBQUM7UUFDWixPQUFPLFFBQVEsQ0FBQztJQUNwQixDQUFDLENBQUMsQ0FBQztBQUNQLENBQUM7QUFFRCxnQkFBZ0I7QUFDaEIsU0FBZ0IscUJBQXFCLENBQ2pDLGFBQWtELEVBQ2xELFlBQXlDO0lBRXpDLElBQUksYUFBYSxLQUFLLFNBQVMsRUFBRSxDQUFDO1FBQzlCLE9BQU87SUFDWCxDQUFDO0lBQ0QsTUFBTSxjQUFjLEdBQUcsSUFBSSxHQUFHLEVBQVUsQ0FBQztJQUN6QyxLQUFLLE1BQU0sS0FBSyxJQUFJLGFBQWEsRUFBRSxDQUFDO1FBQ2hDLE1BQU0sTUFBTSxHQUFHLFlBQVksQ0FBQyxHQUFHLENBQUMsS0FBSyxDQUFDLFlBQVksQ0FBQyxDQUFDO1FBQ3BELElBQUksTUFBTSxLQUFLLFNBQVMsRUFBRSxDQUFDO1lBQ3ZCLE1BQU0sSUFBSSxLQUFLLENBQ1gsNENBQTRDLEtBQUssQ0FBQyxZQUFZLEdBQUcsQ0FDcEUsQ0FBQztRQUNOLENBQUM7UUFDRCxLQUFLLENBQUMsU0FBUyxDQUFDLGtCQUFrQixDQUFDLEtBQUssQ0FBQyxZQUFZLEVBQUUsTUFBTSxDQUFDLElBQUksQ0FBQyxDQUFDO1FBQ3BFLE1BQU0sYUFBYSxHQUFHLEtBQUssQ0FBQyxJQUFJLElBQUksb0JBQW9CLENBQUMsS0FBSyxDQUFDLENBQUM7UUFDaEUsSUFBSSxjQUFjLENBQUMsR0FBRyxDQUFDLGFBQWEsQ0FBQyxFQUFFLENBQUM7WUFDcEMsTUFBTSxJQUFJLEtBQUssQ0FBQyxtQ0FBbUMsYUFBYSxFQUFFLENBQUMsQ0FBQztRQUN4RSxDQUFDO1FBQ0QsY0FBYyxDQUFDLEdBQUcsQ0FBQyxhQUFhLENBQUMsQ0FBQztJQUN0QyxDQUFDO0FBQ0wsQ0FBQztBQUVELGdCQUFnQjtBQUNoQixTQUFnQixpQkFBaUIsQ0FDN0IsU0FBeUMsRUFDekMsWUFBeUM7SUFFekMsSUFBSSxTQUFTLEtBQUssU0FBUyxFQUFFLENBQUM7UUFDMUIsT0FBTztJQUNYLENBQUM7SUFDRCxJQUFJLFNBQVMsQ0FBQyxNQUFNLEtBQUssQ0FBQyxFQUFFLENBQUM7UUFDekIsTUFBTSxJQUFJLEtBQUssQ0FBQyx5REFBeUQsQ0FBQyxDQUFDO0lBQy9FLENBQUM7SUFDRCxLQUFLLE1BQU0sS0FBSyxJQUFJLFNBQVMsRUFBRSxDQUFDO1FBQzVCLE1BQU0sTUFBTSxHQUFHLFlBQVksQ0FBQyxHQUFHLENBQUMsS0FBSyxDQUFDLFlBQVksQ0FBQyxDQUFDO1FBQ3BELElBQUksTUFBTSxLQUFLLFNBQVMsRUFBRSxDQUFDO1lBQ3ZCLE1BQU0sSUFBSSxLQUFLLENBQUMsd0NBQXdDLEtBQUssQ0FBQyxZQUFZLEdBQUcsQ0FBQyxDQUFDO1FBQ25GLENBQUM7UUFDRCxNQUFNLFNBQVMsR0FBRyxLQUFLLENBQUMsU0FBUyxJQUFJLHVEQUF5QixDQUFDLFFBQVEsQ0FBQztRQUN4RSxTQUFTLENBQUMsa0JBQWtCLENBQUMsS0FBSyxDQUFDLFlBQVksRUFBRSxNQUFNLENBQUMsSUFBSSxDQUFDLENBQUM7SUFDbEUsQ0FBQztBQUNMLENBQUM7QUFFRCxnQkFBZ0I7QUFDaEIsU0FBZ0IsbUJBQW1CLENBQy9CLGFBQWtELEVBQ2xELFlBQXlDO0lBRXpDLElBQUksYUFBYSxLQUFLLFNBQVMsSUFBSSxhQUFhLENBQUMsTUFBTSxLQUFLLENBQUMsRUFBRSxDQUFDO1FBQzVELE9BQU8sU0FBUyxDQUFDO0lBQ3JCLENBQUM7SUFDRCxNQUFNLE1BQU0sR0FBNkMsYUFBYSxDQUFDLEdBQUcsQ0FBQyxDQUFDLEtBQUssRUFBRSxLQUFLLEVBQUUsRUFBRSxDQUFDLENBQUM7UUFDMUYsSUFBSSxFQUFFLEtBQUssQ0FBQyxJQUFJLElBQUksb0JBQW9CLENBQUMsS0FBSyxDQUFDO1FBQy9DLFFBQVEsRUFBRSxZQUFZLENBQUMsR0FBRyxDQUFDLEtBQUssQ0FBQyxZQUFZLENBQUUsQ0FBQyxFQUFFO1FBQ2xELFNBQVMsRUFBRSxLQUFLLENBQUMsU0FBUyxDQUFDLGlCQUFpQixFQUFFO1FBQzlDLDREQUE0RDtRQUM1RCw0REFBNEQ7UUFDNUQseURBQXlEO1FBQ3pELDhEQUE4RDtRQUM5RCxtREFBbUQ7UUFDbkQsT0FBTyxFQUFFLElBQUksR0FBRyxLQUFLO0tBQ3hCLENBQUMsQ0FBQyxDQUFDO0lBQ0osT0FBTztRQUNILE1BQU0sRUFBRSxDQUFDO1FBQ1QsTUFBTTtLQUNULENBQUM7QUFDTixDQUFDO0FBRUQsZ0JBQWdCO0FBQ2hCLFNBQWdCLGVBQWUsQ0FDM0IsU0FBeUMsRUFDekMsWUFBeUM7SUFFekMsSUFBSSxTQUFTLEtBQUssU0FBUyxJQUFJLFNBQVMsQ0FBQyxNQUFNLEtBQUssQ0FBQyxFQUFFLENBQUM7UUFDcEQsT0FBTyxTQUFTLENBQUM7SUFDckIsQ0FBQztJQUNELE1BQU0sTUFBTSxHQUF3QyxTQUFTLENBQUMsR0FBRyxDQUFDLENBQUMsS0FBSyxFQUFFLEVBQUU7UUFDeEUsTUFBTSxTQUFTLEdBQUcsS0FBSyxDQUFDLFNBQVMsSUFBSSx1REFBeUIsQ0FBQyxRQUFRLENBQUM7UUFDeEUsT0FBTztZQUNILFFBQVEsRUFBRSxZQUFZLENBQUMsR0FBRyxDQUFDLEtBQUssQ0FBQyxZQUFZLENBQUUsQ0FBQyxFQUFFO1lBQ2xELFNBQVMsRUFBRSxTQUFTLENBQUMsaUJBQWlCLEVBQUU7WUFDeEMsU0FBUyxFQUFFLEtBQUssQ0FBQyxTQUFTLElBQUksb0NBQW9CLENBQUMsR0FBRztZQUN0RCxTQUFTLEVBQUUsS0FBSyxDQUFDLFNBQVMsSUFBSSxnQ0FBZ0IsQ0FBQyxVQUFVO1NBQzVELENBQUM7SUFDTixDQUFDLENBQUMsQ0FBQztJQUNILE9BQU87UUFDSCxPQUFPLEVBQUUsQ0FBQztRQUNWLE1BQU07S0FDVCxDQUFDO0FBQ04sQ0FBQztBQUVELFNBQVMsb0JBQW9CLENBQUMsS0FBNEI7SUFDdEQsTUFBTSxlQUFlLEdBQUcsS0FBSyxDQUFDLFNBQVMsQ0FBQyxpQkFBaUIsRUFBRSxDQUFDO0lBQzVELElBQUksZUFBZSxLQUFLLFVBQVUsRUFBRSxDQUFDO1FBQ2pDLE9BQU8sS0FBSyxDQUFDLFlBQVksQ0FBQztJQUM5QixDQUFDO0lBQ0QseUNBQXlDO0lBQ3pDLE1BQU0sYUFBYSxHQUFHLGVBQWUsQ0FBQyxPQUFPLENBQUMsU0FBUyxFQUFFLEVBQUUsQ0FBQyxDQUFDO0lBQzdELE9BQU8sR0FBRyxLQUFLLENBQUMsWUFBWSxJQUFJLGFBQWEsRUFBRSxDQUFDO0FBQ3BELENBQUM7QUFFRCxnQkFBZ0I7QUFDaEIsU0FBZ0IseUJBQXlCLENBQ3JDLG9CQUEwQyxFQUMxQyxZQUF5QztJQUV6QyxJQUFJLG9CQUFvQixLQUFLLFNBQVMsSUFBSSxvQkFBb0IsQ0FBQyxNQUFNLEtBQUssQ0FBQyxFQUFFLENBQUM7UUFDMUUsT0FBTyxTQUFTLENBQUM7SUFDckIsQ0FBQztJQUNELE1BQU0sSUFBSSxHQUFHLElBQUksR0FBRyxFQUFVLENBQUM7SUFDL0IsT0FBTyxvQkFBb0IsQ0FBQyxHQUFHLENBQUMsQ0FBQyxJQUFJLEVBQUUsRUFBRTtRQUNyQyxJQUFJLElBQUksQ0FBQyxHQUFHLENBQUMsSUFBSSxDQUFDLEVBQUUsQ0FBQztZQUNqQixNQUFNLElBQUksS0FBSyxDQUFDLCtCQUErQixJQUFJLEVBQUUsQ0FBQyxDQUFDO1FBQzNELENBQUM7UUFDRCxJQUFJLENBQUMsR0FBRyxDQUFDLElBQUksQ0FBQyxDQUFDO1FBQ2YsTUFBTSxNQUFNLEdBQUcsWUFBWSxDQUFDLEdBQUcsQ0FBQyxJQUFJLENBQUMsQ0FBQztRQUN0QyxJQUFJLE1BQU0sS0FBSyxTQUFTLEVBQUUsQ0FBQztZQUN2QixNQUFNLElBQUksS0FBSyxDQUFDLG1EQUFtRCxJQUFJLEdBQUcsQ0FBQyxDQUFDO1FBQ2hGLENBQUM7UUFDRCxPQUFPLE1BQU0sQ0FBQyxFQUFFLENBQUM7SUFDckIsQ0FBQyxDQUFDLENBQUM7QUFDUCxDQUFDO0FBRUQsZ0JBQWdCO0FBQ2hCLFNBQWdCLGVBQWUsQ0FDM0IsVUFBNkIsRUFDN0IsYUFBbUMsRUFDbkMsSUFBMkMsRUFDM0MsT0FBMkI7SUFFM0IsSUFBSSxPQUFPLEtBQUssU0FBUyxJQUFJLElBQUksS0FBSyxTQUFTLElBQUksSUFBSSxDQUFDLE9BQU8sS0FBSyxTQUFTLEVBQUUsQ0FBQztRQUM1RSxNQUFNLElBQUksS0FBSyxDQUNYLHdGQUF3RjtjQUN0Riw2QkFBNkIsQ0FDbEMsQ0FBQztJQUNOLENBQUM7SUFDRCxNQUFNLE1BQU0sR0FBOEI7UUFDdEMsQ0FBQyxnREFBcUIsQ0FBQyxjQUFjLENBQUMsRUFBRSxhQUFhO1FBQ3JELENBQUMsZ0RBQXFCLENBQUMsb0JBQW9CLENBQUMsRUFBRSxVQUFVO1FBQ3hELEdBQUcsQ0FBQyxJQUFJLElBQUksRUFBRSxDQUFDO0tBQ2xCLENBQUM7SUFDRixJQUFJLE9BQU8sS0FBSyxTQUFTLEVBQUUsQ0FBQztRQUN4QixNQUFNLENBQUMsT0FBTyxHQUFHLE9BQU8sQ0FBQztJQUM3QixDQUFDO0lBQ0QsT0FBTyxNQUFNLENBQUM7QUFDbEIsQ0FBQyIsInNvdXJjZXNDb250ZW50IjpbImltcG9ydCB7XG4gICAgQ2ZuVGFibGUsXG59IGZyb20gJ2F3cy1jZGstbGliL2F3cy1nbHVlJztcbmltcG9ydCB7XG4gICAgSWNlYmVyZ1JlbmRlckNvbnRleHQsXG4gICAgSWNlYmVyZ1R5cGUsXG59IGZyb20gJy4vaWNlYmVyZy10eXBlJztcbmltcG9ydCB7XG4gICAgSWNlYmVyZ1BhcnRpdGlvblRyYW5zZm9ybSxcbn0gZnJvbSAnLi9pY2ViZXJnLXBhcnRpdGlvbi10cmFuc2Zvcm0nO1xuaW1wb3J0IHtcbiAgICBJQ0VCRVJHX1BST1BFUlRZX0tFWVMsXG4gICAgSWNlYmVyZ0RhdGFGb3JtYXQsXG4gICAgSWNlYmVyZ0Zvcm1hdFZlcnNpb24sXG59IGZyb20gJy4vaWNlYmVyZy10YWJsZS1wcm9wZXJ0aWVzJztcbmltcG9ydCB7XG4gICAgSWNlYmVyZ0NvbHVtbixcbiAgICBJY2ViZXJnTnVsbE9yZGVyLFxuICAgIEljZWJlcmdQYXJ0aXRpb25GaWVsZCxcbiAgICBJY2ViZXJnU29ydERpcmVjdGlvbixcbiAgICBJY2ViZXJnU29ydEZpZWxkLFxuICAgIEljZWJlcmdUYWJsZVByb3BzLFxufSBmcm9tICcuL2ljZWJlcmctdGFibGUnO1xuXG4vLy8gLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vL1xuLy8vIC8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy9cbi8vIFZhbGlkYXRpb25cblxuLyoqIEBpbnRlcm5hbCAqL1xuZXhwb3J0IGZ1bmN0aW9uIHZhbGlkYXRlUHJvcHMocHJvcHM6IEljZWJlcmdUYWJsZVByb3BzKTogdm9pZCB7XG4gICAgaWYgKCFwcm9wcy50YWJsZU5hbWUubWF0Y2goL15bYS16MC05X10rJC8pKSB7XG4gICAgICAgIHRocm93IG5ldyBFcnJvcihcbiAgICAgICAgICAgIGB0YWJsZU5hbWUgJyR7cHJvcHMudGFibGVOYW1lfScgbXVzdCBjb250YWluIG9ubHkgbG93ZXJjYXNlIGxldHRlcnMsIGRpZ2l0cywgYW5kIHVuZGVyc2NvcmVzYCxcbiAgICAgICAgKTtcbiAgICB9XG4gICAgaWYgKHByb3BzLmNvbHVtbnMubGVuZ3RoID09PSAwKSB7XG4gICAgICAgIHRocm93IG5ldyBFcnJvcignSWNlYmVyZ1RhYmxlIHJlcXVpcmVzIGF0IGxlYXN0IG9uZSBjb2x1bW4nKTtcbiAgICB9XG4gICAgY29uc3QgbmFtZXMgPSBuZXcgU2V0PHN0cmluZz4oKTtcbiAgICBmb3IgKGNvbnN0IGNvbHVtbiBvZiBwcm9wcy5jb2x1bW5zKSB7XG4gICAgICAgIGlmIChuYW1lcy5oYXMoY29sdW1uLm5hbWUpKSB7XG4gICAgICAgICAgICB0aHJvdyBuZXcgRXJyb3IoYGR1cGxpY2F0ZSBjb2x1bW4gbmFtZTogJHtjb2x1bW4ubmFtZX1gKTtcbiAgICAgICAgfVxuICAgICAgICBuYW1lcy5hZGQoY29sdW1uLm5hbWUpO1xuICAgIH1cbiAgICBpZiAoIXByb3BzLmxvY2F0aW9uLnN0YXJ0c1dpdGgoJ3MzOi8vJykpIHtcbiAgICAgICAgdGhyb3cgbmV3IEVycm9yKGBsb2NhdGlvbiBtdXN0IHN0YXJ0IHdpdGggJ3MzOi8vJywgZ290ICcke3Byb3BzLmxvY2F0aW9ufSdgKTtcbiAgICB9XG59XG5cbi8qKiBAaW50ZXJuYWwgKi9cbmV4cG9ydCBmdW5jdGlvbiBub3JtYWxpemVMb2NhdGlvbihsb2NhdGlvbjogc3RyaW5nKTogc3RyaW5nIHtcbiAgICByZXR1cm4gbG9jYXRpb24uZW5kc1dpdGgoJy8nKSA/IGxvY2F0aW9uIDogYCR7bG9jYXRpb259L2A7XG59XG5cbi8qKiBAaW50ZXJuYWwgKi9cbmV4cG9ydCBmdW5jdGlvbiBwYXJzZVMzVXJpKHVyaTogc3RyaW5nKTogeyBidWNrZXQ6IHN0cmluZzsga2V5OiBzdHJpbmcgfSB7XG4gICAgLy8vIGBub3JtYWxpemVMb2NhdGlvbmAgaXMgYWx3YXlzIGNhbGxlZCBmaXJzdCwgc28gYHVyaWAgZW5kcyB3aXRoIGAvYFxuICAgIC8vLyBhbmQgdGhlcmVmb3JlIGNvbnRhaW5zIGF0IGxlYXN0IG9uZSBgL2AgYWZ0ZXIgdGhlIHNjaGVtZS5cbiAgICBjb25zdCByZXN0ID0gdXJpLnN1YnN0cmluZygnczM6Ly8nLmxlbmd0aCk7XG4gICAgY29uc3Qgc2xhc2ggPSByZXN0LmluZGV4T2YoJy8nKTtcbiAgICByZXR1cm4ge1xuICAgICAgICBidWNrZXQ6IHJlc3Quc3Vic3RyaW5nKDAsIHNsYXNoKSxcbiAgICAgICAga2V5OiByZXN0LnN1YnN0cmluZyhzbGFzaCArIDEpLFxuICAgIH07XG59XG5cbi8vLyAvLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vXG4vLy8gLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vLy8vL1xuLy8gU2NoZW1hIC8gcGFydGl0aW9uIC8gc29ydCByZW5kZXJpbmdcblxuaW50ZXJmYWNlIFJlbmRlcmVkQ29sdW1uIHtcbiAgICByZWFkb25seSBpZDogbnVtYmVyO1xuICAgIHJlYWRvbmx5IHR5cGU6IEljZWJlcmdUeXBlO1xufVxuXG4vKiogQGludGVybmFsICovXG5leHBvcnQgaW50ZXJmYWNlIFJlbmRlcmVkU2NoZW1hIHtcbiAgICByZWFkb25seSBmaWVsZHM6IENmblRhYmxlLkljZWJlcmdTdHJ1Y3RGaWVsZFByb3BlcnR5W107XG4gICAgcmVhZG9ubHkgY29sdW1uQnlOYW1lOiBNYXA8c3RyaW5nLCBSZW5kZXJlZENvbHVtbj47XG59XG5cbi8qKiBAaW50ZXJuYWwgKi9cbmV4cG9ydCBmdW5jdGlvbiByZW5kZXJTY2hlbWEoY29sdW1uczogSWNlYmVyZ0NvbHVtbltdKTogUmVuZGVyZWRTY2hlbWEge1xuICAgIC8vLyBGaXJzdCBwYXNzOiBhc3NpZ24gdG9wLWxldmVsIGlkcy4gUGlubmVkIGlkcyB3aW47IHRoZSByZXN0IGFyZVxuICAgIC8vLyBmaWxsZWQgd2l0aCB0aGUgc21hbGxlc3QgdW51c2VkIHBvc2l0aXZlIGludGVnZXJzLiBOZXN0ZWQtdHlwZVxuICAgIC8vLyBpZHMgY29udGludWUgYWJvdmUgdGhlIGhpZ2hlc3QgdG9wLWxldmVsIGlkIHNvIHRoZXkgbmV2ZXJcbiAgICAvLy8gY29sbGlkZSB3aXRoIGNvbHVtbiBpZHMuXG4gICAgY29uc3QgdG9wTGV2ZWxJZHMgPSBhc3NpZ25Ub3BMZXZlbElkcyhjb2x1bW5zKTtcbiAgICBsZXQgbmV4dElkID0gTWF0aC5tYXgoLi4udG9wTGV2ZWxJZHMsIGNvbHVtbnMubGVuZ3RoKSArIDE7XG4gICAgY29uc3QgY3R4OiBJY2ViZXJnUmVuZGVyQ29udGV4dCA9IHtcbiAgICAgICAgbmV4dElkOiAoKSA9PiB7XG4gICAgICAgICAgICBjb25zdCBpZCA9IG5leHRJZDtcbiAgICAgICAgICAgIG5leHRJZCArPSAxO1xuICAgICAgICAgICAgcmV0dXJuIGlkO1xuICAgICAgICB9LFxuICAgIH07XG4gICAgY29uc3QgY29sdW1uQnlOYW1lID0gbmV3IE1hcDxzdHJpbmcsIFJlbmRlcmVkQ29sdW1uPigpO1xuICAgIGNvbnN0IGZpZWxkczogQ2ZuVGFibGUuSWNlYmVyZ1N0cnVjdEZpZWxkUHJvcGVydHlbXSA9IGNvbHVtbnMubWFwKChjb2x1bW4sIGluZGV4KSA9PiB7XG4gICAgICAgIGNvbnN0IGlkID0gdG9wTGV2ZWxJZHNbaW5kZXhdO1xuICAgICAgICBjb25zdCB0eXBlU3RyaW5nID0gY29sdW1uLnR5cGUuX3JlbmRlcihjdHgpO1xuICAgICAgICBjb2x1bW5CeU5hbWUuc2V0KGNvbHVtbi5uYW1lLCB7XG4gICAgICAgICAgICBpZCxcbiAgICAgICAgICAgIHR5cGU6IGNvbHVtbi50eXBlLFxuICAgICAgICB9KTtcbiAgICAgICAgcmV0dXJuIHtcbiAgICAgICAgICAgIGlkLFxuICAgICAgICAgICAgbmFtZTogY29sdW1uLm5hbWUsXG4gICAgICAgICAgICB0eXBlOiB0eXBlU3RyaW5nLFxuICAgICAgICAgICAgcmVxdWlyZWQ6IGNvbHVtbi5yZXF1aXJlZCA/PyBmYWxzZSxcbiAgICAgICAgICAgIGRvYzogY29sdW1uLmRvYyxcbiAgICAgICAgfTtcbiAgICB9KTtcbiAgICByZXR1cm4ge1xuICAgICAgICBmaWVsZHMsXG4gICAgICAgIGNvbHVtbkJ5TmFtZSxcbiAgICB9O1xufVxuXG4vKipcbiAqIEFzc2lnbiB0b3AtbGV2ZWwgZmllbGQgaWRzIGhvbm9yaW5nIHVzZXItcGlubmVkIGlkcyBhbmQgZmlsbGluZ1xuICogdW5zcGVjaWZpZWQgb25lcyB3aXRoIHRoZSBzbWFsbGVzdCB1bnVzZWQgcG9zaXRpdmUgaW50ZWdlci5cbiAqIFRocm93cyBvbiBkdXBsaWNhdGUgb3Igbm9uLXBvc2l0aXZlIGlkcy5cbiAqL1xuZnVuY3Rpb24gYXNzaWduVG9wTGV2ZWxJZHMoY29sdW1uczogSWNlYmVyZ0NvbHVtbltdKTogbnVtYmVyW10ge1xuICAgIGNvbnN0IHRha2VuID0gbmV3IFNldDxudW1iZXI+KCk7XG4gICAgZm9yIChjb25zdCBjb2x1bW4gb2YgY29sdW1ucykge1xuICAgICAgICBpZiAoY29sdW1uLmlkID09PSB1bmRlZmluZWQpIHtcbiAgICAgICAgICAgIGNvbnRpbnVlO1xuICAgICAgICB9XG4gICAgICAgIGlmICghTnVtYmVyLmlzSW50ZWdlcihjb2x1bW4uaWQpIHx8IGNvbHVtbi5pZCA8IDEpIHtcbiAgICAgICAgICAgIHRocm93IG5ldyBFcnJvcihcbiAgICAgICAgICAgICAgICBgY29sdW1uICcke2NvbHVtbi5uYW1lfScgaGFzIGludmFsaWQgaWQgJHtjb2x1bW4uaWR9OyBpZHMgbXVzdCBiZSBwb3NpdGl2ZSBpbnRlZ2Vyc2AsXG4gICAgICAgICAgICApO1xuICAgICAgICB9XG4gICAgICAgIGlmICh0YWtlbi5oYXMoY29sdW1uLmlkKSkge1xuICAgICAgICAgICAgdGhyb3cgbmV3IEVycm9yKGBkdXBsaWNhdGUgY29sdW1uIGlkICR7Y29sdW1uLmlkfSBvbiBjb2x1bW4gJyR7Y29sdW1uLm5hbWV9J2ApO1xuICAgICAgICB9XG4gICAgICAgIHRha2VuLmFkZChjb2x1bW4uaWQpO1xuICAgIH1cbiAgICBsZXQgY3Vyc29yID0gMTtcbiAgICByZXR1cm4gY29sdW1ucy5tYXAoKGNvbHVtbikgPT4ge1xuICAgICAgICBpZiAoY29sdW1uLmlkICE9PSB1bmRlZmluZWQpIHtcbiAgICAgICAgICAgIHJldHVybiBjb2x1bW4uaWQ7XG4gICAgICAgIH1cbiAgICAgICAgd2hpbGUgKHRha2VuLmhhcyhjdXJzb3IpKSB7XG4gICAgICAgICAgICBjdXJzb3IgKz0gMTtcbiAgICAgICAgfVxuICAgICAgICB0YWtlbi5hZGQoY3Vyc29yKTtcbiAgICAgICAgY29uc3QgYXNzaWduZWQgPSBjdXJzb3I7XG4gICAgICAgIGN1cnNvciArPSAxO1xuICAgICAgICByZXR1cm4gYXNzaWduZWQ7XG4gICAgfSk7XG59XG5cbi8qKiBAaW50ZXJuYWwgKi9cbmV4cG9ydCBmdW5jdGlvbiB2YWxpZGF0ZVBhcnRpdGlvblNwZWMoXG4gICAgcGFydGl0aW9uU3BlYzogSWNlYmVyZ1BhcnRpdGlvbkZpZWxkW10gfCB1bmRlZmluZWQsXG4gICAgY29sdW1uQnlOYW1lOiBNYXA8c3RyaW5nLCBSZW5kZXJlZENvbHVtbj4sXG4pOiB2b2lkIHtcbiAgICBpZiAocGFydGl0aW9uU3BlYyA9PT0gdW5kZWZpbmVkKSB7XG4gICAgICAgIHJldHVybjtcbiAgICB9XG4gICAgY29uc3QgcGFydGl0aW9uTmFtZXMgPSBuZXcgU2V0PHN0cmluZz4oKTtcbiAgICBmb3IgKGNvbnN0IGZpZWxkIG9mIHBhcnRpdGlvblNwZWMpIHtcbiAgICAgICAgY29uc3Qgc291cmNlID0gY29sdW1uQnlOYW1lLmdldChmaWVsZC5zb3VyY2VDb2x1bW4pO1xuICAgICAgICBpZiAoc291cmNlID09PSB1bmRlZmluZWQpIHtcbiAgICAgICAgICAgIHRocm93IG5ldyBFcnJvcihcbiAgICAgICAgICAgICAgICBgcGFydGl0aW9uU3BlYyByZWZlcmVuY2VzIHVua25vd24gY29sdW1uICcke2ZpZWxkLnNvdXJjZUNvbHVtbn0nYCxcbiAgICAgICAgICAgICk7XG4gICAgICAgIH1cbiAgICAgICAgZmllbGQudHJhbnNmb3JtLnZhbGlkYXRlU291cmNlVHlwZShmaWVsZC5zb3VyY2VDb2x1bW4sIHNvdXJjZS50eXBlKTtcbiAgICAgICAgY29uc3QgcGFydGl0aW9uTmFtZSA9IGZpZWxkLm5hbWUgPz8gZGVmYXVsdFBhcnRpdGlvbk5hbWUoZmllbGQpO1xuICAgICAgICBpZiAocGFydGl0aW9uTmFtZXMuaGFzKHBhcnRpdGlvbk5hbWUpKSB7XG4gICAgICAgICAgICB0aHJvdyBuZXcgRXJyb3IoYGR1cGxpY2F0ZSBwYXJ0aXRpb24gZmllbGQgbmFtZTogJHtwYXJ0aXRpb25OYW1lfWApO1xuICAgICAgICB9XG4gICAgICAgIHBhcnRpdGlvbk5hbWVzLmFkZChwYXJ0aXRpb25OYW1lKTtcbiAgICB9XG59XG5cbi8qKiBAaW50ZXJuYWwgKi9cbmV4cG9ydCBmdW5jdGlvbiB2YWxpZGF0ZVNvcnRPcmRlcihcbiAgICBzb3J0T3JkZXI6IEljZWJlcmdTb3J0RmllbGRbXSB8IHVuZGVmaW5lZCxcbiAgICBjb2x1bW5CeU5hbWU6IE1hcDxzdHJpbmcsIFJlbmRlcmVkQ29sdW1uPixcbik6IHZvaWQge1xuICAgIGlmIChzb3J0T3JkZXIgPT09IHVuZGVmaW5lZCkge1xuICAgICAgICByZXR1cm47XG4gICAgfVxuICAgIGlmIChzb3J0T3JkZXIubGVuZ3RoID09PSAwKSB7XG4gICAgICAgIHRocm93IG5ldyBFcnJvcignc29ydE9yZGVyIG11c3QgY29udGFpbiBhdCBsZWFzdCBvbmUgZmllbGQgd2hlbiBwcm92aWRlZCcpO1xuICAgIH1cbiAgICBmb3IgKGNvbnN0IGZpZWxkIG9mIHNvcnRPcmRlcikge1xuICAgICAgICBjb25zdCBzb3VyY2UgPSBjb2x1bW5CeU5hbWUuZ2V0KGZpZWxkLnNvdXJjZUNvbHVtbik7XG4gICAgICAgIGlmIChzb3VyY2UgPT09IHVuZGVmaW5lZCkge1xuICAgICAgICAgICAgdGhyb3cgbmV3IEVycm9yKGBzb3J0T3JkZXIgcmVmZXJlbmNlcyB1bmtub3duIGNvbHVtbiAnJHtmaWVsZC5zb3VyY2VDb2x1bW59J2ApO1xuICAgICAgICB9XG4gICAgICAgIGNvbnN0IHRyYW5zZm9ybSA9IGZpZWxkLnRyYW5zZm9ybSA/PyBJY2ViZXJnUGFydGl0aW9uVHJhbnNmb3JtLklERU5USVRZO1xuICAgICAgICB0cmFuc2Zvcm0udmFsaWRhdGVTb3VyY2VUeXBlKGZpZWxkLnNvdXJjZUNvbHVtbiwgc291cmNlLnR5cGUpO1xuICAgIH1cbn1cblxuLyoqIEBpbnRlcm5hbCAqL1xuZXhwb3J0IGZ1bmN0aW9uIHJlbmRlclBhcnRpdGlvblNwZWMoXG4gICAgcGFydGl0aW9uU3BlYzogSWNlYmVyZ1BhcnRpdGlvbkZpZWxkW10gfCB1bmRlZmluZWQsXG4gICAgY29sdW1uQnlOYW1lOiBNYXA8c3RyaW5nLCBSZW5kZXJlZENvbHVtbj4sXG4pOiBDZm5UYWJsZS5JY2ViZXJnUGFydGl0aW9uU3BlY1Byb3BlcnR5IHwgdW5kZWZpbmVkIHtcbiAgICBpZiAocGFydGl0aW9uU3BlYyA9PT0gdW5kZWZpbmVkIHx8IHBhcnRpdGlvblNwZWMubGVuZ3RoID09PSAwKSB7XG4gICAgICAgIHJldHVybiB1bmRlZmluZWQ7XG4gICAgfVxuICAgIGNvbnN0IGZpZWxkczogQ2ZuVGFibGUuSWNlYmVyZ1BhcnRpdGlvbkZpZWxkUHJvcGVydHlbXSA9IHBhcnRpdGlvblNwZWMubWFwKChmaWVsZCwgaW5kZXgpID0+ICh7XG4gICAgICAgIG5hbWU6IGZpZWxkLm5hbWUgPz8gZGVmYXVsdFBhcnRpdGlvbk5hbWUoZmllbGQpLFxuICAgICAgICBzb3VyY2VJZDogY29sdW1uQnlOYW1lLmdldChmaWVsZC5zb3VyY2VDb2x1bW4pIS5pZCxcbiAgICAgICAgdHJhbnNmb3JtOiBmaWVsZC50cmFuc2Zvcm0udG9UcmFuc2Zvcm1TdHJpbmcoKSxcbiAgICAgICAgLy8vIFBlciB0aGUgSWNlYmVyZyBzcGVjLCBwYXJ0aXRpb24gZmllbGQgaWRzIG11c3QgYmUgaW4gdGhlXG4gICAgICAgIC8vLyByYW5nZSBbMTAwMCwgOTk5OV07IHRoZSBjb25zdHJ1Y3QgYWxsb2NhdGVzIHRoZW0gZGVuc2VseVxuICAgICAgICAvLy8gZnJvbSAxMDAwIGluIGRlY2xhcmF0aW9uIG9yZGVyLiBSZW9yZGVyaW5nIHBhcnRpdGlvbnNcbiAgICAgICAgLy8vIGFjcm9zcyBkZXBsb3lzIHJlYXNzaWducyBpZHMg4oCUIHNlZSB0aGUgXCJwYXJ0aXRpb24gZmllbGQgaWRcbiAgICAgICAgLy8vIHJldXNlXCIgZW50cnkgaW4gdGhlIFJFQURNRSdzIEtub3duIGxpbWl0YXRpb25zLlxuICAgICAgICBmaWVsZElkOiAxMDAwICsgaW5kZXgsXG4gICAgfSkpO1xuICAgIHJldHVybiB7XG4gICAgICAgIHNwZWNJZDogMCxcbiAgICAgICAgZmllbGRzLFxuICAgIH07XG59XG5cbi8qKiBAaW50ZXJuYWwgKi9cbmV4cG9ydCBmdW5jdGlvbiByZW5kZXJTb3J0T3JkZXIoXG4gICAgc29ydE9yZGVyOiBJY2ViZXJnU29ydEZpZWxkW10gfCB1bmRlZmluZWQsXG4gICAgY29sdW1uQnlOYW1lOiBNYXA8c3RyaW5nLCBSZW5kZXJlZENvbHVtbj4sXG4pOiBDZm5UYWJsZS5JY2ViZXJnU29ydE9yZGVyUHJvcGVydHkgfCB1bmRlZmluZWQge1xuICAgIGlmIChzb3J0T3JkZXIgPT09IHVuZGVmaW5lZCB8fCBzb3J0T3JkZXIubGVuZ3RoID09PSAwKSB7XG4gICAgICAgIHJldHVybiB1bmRlZmluZWQ7XG4gICAgfVxuICAgIGNvbnN0IGZpZWxkczogQ2ZuVGFibGUuSWNlYmVyZ1NvcnRGaWVsZFByb3BlcnR5W10gPSBzb3J0T3JkZXIubWFwKChmaWVsZCkgPT4ge1xuICAgICAgICBjb25zdCB0cmFuc2Zvcm0gPSBmaWVsZC50cmFuc2Zvcm0gPz8gSWNlYmVyZ1BhcnRpdGlvblRyYW5zZm9ybS5JREVOVElUWTtcbiAgICAgICAgcmV0dXJuIHtcbiAgICAgICAgICAgIHNvdXJjZUlkOiBjb2x1bW5CeU5hbWUuZ2V0KGZpZWxkLnNvdXJjZUNvbHVtbikhLmlkLFxuICAgICAgICAgICAgdHJhbnNmb3JtOiB0cmFuc2Zvcm0udG9UcmFuc2Zvcm1TdHJpbmcoKSxcbiAgICAgICAgICAgIGRpcmVjdGlvbjogZmllbGQuZGlyZWN0aW9uID8/IEljZWJlcmdTb3J0RGlyZWN0aW9uLkFTQyxcbiAgICAgICAgICAgIG51bGxPcmRlcjogZmllbGQubnVsbE9yZGVyID8/IEljZWJlcmdOdWxsT3JkZXIuTlVMTFNfTEFTVCxcbiAgICAgICAgfTtcbiAgICB9KTtcbiAgICByZXR1cm4ge1xuICAgICAgICBvcmRlcklkOiAxLFxuICAgICAgICBmaWVsZHMsXG4gICAgfTtcbn1cblxuZnVuY3Rpb24gZGVmYXVsdFBhcnRpdGlvbk5hbWUoZmllbGQ6IEljZWJlcmdQYXJ0aXRpb25GaWVsZCk6IHN0cmluZyB7XG4gICAgY29uc3QgdHJhbnNmb3JtU3RyaW5nID0gZmllbGQudHJhbnNmb3JtLnRvVHJhbnNmb3JtU3RyaW5nKCk7XG4gICAgaWYgKHRyYW5zZm9ybVN0cmluZyA9PT0gJ2lkZW50aXR5Jykge1xuICAgICAgICByZXR1cm4gZmllbGQuc291cmNlQ29sdW1uO1xuICAgIH1cbiAgICAvLy8gYGJ1Y2tldFsxNl1gIC0+IGBzb3VyY2VDb2x1bW5fYnVja2V0YFxuICAgIGNvbnN0IGJhc2VUcmFuc2Zvcm0gPSB0cmFuc2Zvcm1TdHJpbmcucmVwbGFjZSgvXFxbLipcXF0kLywgJycpO1xuICAgIHJldHVybiBgJHtmaWVsZC5zb3VyY2VDb2x1bW59XyR7YmFzZVRyYW5zZm9ybX1gO1xufVxuXG4vKiogQGludGVybmFsICovXG5leHBvcnQgZnVuY3Rpb24gcmVzb2x2ZUlkZW50aWZpZXJGaWVsZElkcyhcbiAgICBpZGVudGlmaWVyRmllbGROYW1lczogc3RyaW5nW10gfCB1bmRlZmluZWQsXG4gICAgY29sdW1uQnlOYW1lOiBNYXA8c3RyaW5nLCBSZW5kZXJlZENvbHVtbj4sXG4pOiBudW1iZXJbXSB8IHVuZGVmaW5lZCB7XG4gICAgaWYgKGlkZW50aWZpZXJGaWVsZE5hbWVzID09PSB1bmRlZmluZWQgfHwgaWRlbnRpZmllckZpZWxkTmFtZXMubGVuZ3RoID09PSAwKSB7XG4gICAgICAgIHJldHVybiB1bmRlZmluZWQ7XG4gICAgfVxuICAgIGNvbnN0IHNlZW4gPSBuZXcgU2V0PHN0cmluZz4oKTtcbiAgICByZXR1cm4gaWRlbnRpZmllckZpZWxkTmFtZXMubWFwKChuYW1lKSA9PiB7XG4gICAgICAgIGlmIChzZWVuLmhhcyhuYW1lKSkge1xuICAgICAgICAgICAgdGhyb3cgbmV3IEVycm9yKGBkdXBsaWNhdGUgaWRlbnRpZmllciBmaWVsZDogJHtuYW1lfWApO1xuICAgICAgICB9XG4gICAgICAgIHNlZW4uYWRkKG5hbWUpO1xuICAgICAgICBjb25zdCBjb2x1bW4gPSBjb2x1bW5CeU5hbWUuZ2V0KG5hbWUpO1xuICAgICAgICBpZiAoY29sdW1uID09PSB1bmRlZmluZWQpIHtcbiAgICAgICAgICAgIHRocm93IG5ldyBFcnJvcihgaWRlbnRpZmllckZpZWxkTmFtZXMgcmVmZXJlbmNlcyB1bmtub3duIGNvbHVtbiAnJHtuYW1lfSdgKTtcbiAgICAgICAgfVxuICAgICAgICByZXR1cm4gY29sdW1uLmlkO1xuICAgIH0pO1xufVxuXG4vKiogQGludGVybmFsICovXG5leHBvcnQgZnVuY3Rpb24gbWVyZ2VQcm9wZXJ0aWVzKFxuICAgIGRhdGFGb3JtYXQ6IEljZWJlcmdEYXRhRm9ybWF0LFxuICAgIGZvcm1hdFZlcnNpb246IEljZWJlcmdGb3JtYXRWZXJzaW9uLFxuICAgIHVzZXI6IHsgW2tleTogc3RyaW5nXTogc3RyaW5nIH0gfCB1bmRlZmluZWQsXG4gICAgY29tbWVudDogc3RyaW5nIHwgdW5kZWZpbmVkLFxuKTogeyBba2V5OiBzdHJpbmddOiBzdHJpbmcgfSB7XG4gICAgaWYgKGNvbW1lbnQgIT09IHVuZGVmaW5lZCAmJiB1c2VyICE9PSB1bmRlZmluZWQgJiYgdXNlci5jb21tZW50ICE9PSB1bmRlZmluZWQpIHtcbiAgICAgICAgdGhyb3cgbmV3IEVycm9yKFxuICAgICAgICAgICAgJ2NvbW1lbnQgaXMgc2V0IGJvdGggYXMgdGhlIHRvcC1sZXZlbCBgY29tbWVudGAgcHJvcCBhbmQgYXMgYHRhYmxlUHJvcGVydGllcy5jb21tZW50YDsgJ1xuICAgICAgICAgICAgKyAnc2V0IGl0IGluIGV4YWN0bHkgb25lIHBsYWNlJyxcbiAgICAgICAgKTtcbiAgICB9XG4gICAgY29uc3QgbWVyZ2VkOiB7IFtrZXk6IHN0cmluZ106IHN0cmluZyB9ID0ge1xuICAgICAgICBbSUNFQkVSR19QUk9QRVJUWV9LRVlTLkZPUk1BVF9WRVJTSU9OXTogZm9ybWF0VmVyc2lvbixcbiAgICAgICAgW0lDRUJFUkdfUFJPUEVSVFlfS0VZUy5XUklURV9GT1JNQVRfREVGQVVMVF06IGRhdGFGb3JtYXQsXG4gICAgICAgIC4uLih1c2VyID8/IHt9KSxcbiAgICB9O1xuICAgIGlmIChjb21tZW50ICE9PSB1bmRlZmluZWQpIHtcbiAgICAgICAgbWVyZ2VkLmNvbW1lbnQgPSBjb21tZW50O1xuICAgIH1cbiAgICByZXR1cm4gbWVyZ2VkO1xufVxuIl19
|
|
@@ -96,6 +96,8 @@ export interface IcebergTableProps {
|
|
|
96
96
|
* Extra `properties` to publish on the table. Auto-added keys
|
|
97
97
|
* (`format-version`, `write.format.default`) are merged in and
|
|
98
98
|
* must agree with `formatVersion` / `dataFormat` if user-supplied.
|
|
99
|
+
* Setting `comment` here AND in the top-level `comment` prop
|
|
100
|
+
* throws at synth time — pick one.
|
|
99
101
|
*/
|
|
100
102
|
readonly tableProperties?: {
|
|
101
103
|
[key: string]: string;
|
|
@@ -107,6 +109,7 @@ export interface IcebergTableProps {
|
|
|
107
109
|
* Glue field. The comment is published in the Iceberg
|
|
108
110
|
* `properties` map under the `comment` key, where Athena's
|
|
109
111
|
* `SHOW TBLPROPERTIES` and Iceberg-aware readers can find it.
|
|
112
|
+
* Mutually exclusive with `tableProperties['comment']`.
|
|
110
113
|
*/
|
|
111
114
|
readonly comment?: string;
|
|
112
115
|
/**
|