cdk-glue-iceberg-table 0.1.5 → 0.1.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (2) hide show
  1. package/README.md +68 -18
  2. package/package.json +1 -1
package/README.md CHANGED
@@ -76,6 +76,13 @@ This repo is **both** the published package and a CDK demo app:
76
76
 
77
77
  - `lib/iceberg/` — the published package (`cdk-glue-iceberg-table` on npm).
78
78
  - `lib/arceus-stack.ts`, `lib/iceberg-evolution-stack.ts`, `bin/`, `scripts/` — a CDK app that dogfoods the construct against a real AWS account, plus a bash harness that drives schema + partition evolution through real `cdk deploy`s. **Repo-only, not published to npm.**
79
+ - `e2e-consumer/` — a standalone CDK app that depends on the **published** `cdk-glue-iceberg-table` from npm. Proves that a fresh install + import + `cdk synth` works for downstream consumers. Runs on every PR via the `e2e-consumer` job in `.github/workflows/ci.yml`. Its `lib/surface-reference.ts` touches every exported symbol so that a rename anywhere in the published surface breaks CI.
80
+
81
+ How the test gates fit together:
82
+
83
+ - **`ci.yml`** runs on every PR — lint, unit tests with 95% coverage gate, `npm pack`, and the `e2e-consumer` synth against the latest npm version.
84
+ - **`integ-test.yml`** is the real-AWS gate (`scripts/integration-test-evolution.sh` driving four `cdk deploy`s). Gated by `run-integ-test` label or `/run-integ-test` collaborator comment. PRs that touch any file under `lib/`, `bin/arceus.ts`, `cdk.json`, or the script must show a green run before merging (see CLAUDE.md §"Integration test for construct-touching PRs"). Doc-only PRs are exempt.
85
+ - **`publish.yml`** runs on push to `main` — trusted-publish to npm when `package.json`'s `version` is newer than the registry.
79
86
 
80
87
  The sections [Prerequisites](#prerequisites), [Quickstart](#quickstart),
81
88
  [Demo tables](#demo-tables-deployed-by-arceusstack), and
@@ -92,11 +99,26 @@ Before running the quickstart you need:
92
99
  2. `CDK_DEFAULT_ACCOUNT` and `CDK_DEFAULT_REGION` set in the
93
100
  environment (the AWS CLI sets these automatically for most
94
101
  profile setups; `cdk` also populates them from the active profile).
95
- 3. `DEVELOPER_IAM_USER` set to the name of an **existing IAM user**
96
- in this account. The stack adds that user as a Lake Formation
97
- admin and grants it per-table `SELECT/INSERT/DELETE/ALTER/DESCRIBE`
98
- on the demo Iceberg tables — without it the deploy fails when LF
99
- can't resolve the principal.
102
+ 3. `PRINCIPAL_ARN` set to the **ARN of an existing IAM principal**
103
+ in this account — IAM user, role, or federated identity. The
104
+ stack adds that principal as a Lake Formation admin and grants
105
+ it per-table `SELECT/INSERT/DELETE/ALTER/DESCRIBE` on the demo
106
+ Iceberg tables. Without it the deploy fails when LF can't resolve
107
+ the principal. The same ARN must also be the identity running
108
+ `cdk deploy` and any subsequent Athena queries — local devs
109
+ typically set this to their IAM user ARN; CI (`integ-test.yml`)
110
+ sets it to the OIDC role ARN.
111
+ - **Only one principal is privileged at a time.** `ArceusStack`
112
+ writes the Lake Formation admin list with REPLACE semantics,
113
+ so the most recent `cdk deploy` wins. A local `cdk deploy` with
114
+ your user ARN revokes the OIDC role's LF grants (and breaks the
115
+ next integ-test run until CI redeploys), and vice versa.
116
+ Coordinate accordingly.
117
+ - **SSO / `aws-vault` users:** set `PRINCIPAL_ARN` to the
118
+ canonical role ARN (`arn:aws:iam::<acct>:role/<RoleName>`), not
119
+ the per-session `arn:aws:sts::<acct>:assumed-role/...` you'd
120
+ get from `aws sts get-caller-identity`. LF grants on the
121
+ session-suffixed ARN go stale at the next SSO refresh.
100
122
  4. The Lake Formation service-linked role
101
123
  `AWSServiceRoleForLakeFormationDataAccess` must exist in the
102
124
  account. Create it once with
@@ -119,13 +141,23 @@ Repo source paths (npm consumers import from the package root and get the same e
119
141
  ## Quickstart
120
142
 
121
143
  ```bash
122
- # DEVELOPER_IAM_USER must be the name of an existing IAM user in
123
- # this account. `aws iam get-user` works when you're authenticated
124
- # directly as an IAM user; for SSO / assumed-role / aws-vault setups
125
- # (the modern default) the call fails with "Must specify userName
126
- # when calling with non-User credentials" and you should set the
127
- # variable manually to whichever IAM user you want LF to grant on.
128
- export DEVELOPER_IAM_USER="$(aws iam get-user --query 'User.UserName' --output text)"
144
+ # PRINCIPAL_ARN is the ARN of the IAM principal (user, role, or
145
+ # federated identity) that the stack should make a Lake Formation
146
+ # admin and per-table grantee. It must equal the identity running
147
+ # `cdk deploy` and any subsequent Athena queries — otherwise the
148
+ # integration script's INSERT/SELECT calls fail with `Principal does
149
+ # not have any privilege on specified resource`.
150
+ #
151
+ # For a direct IAM-user session, the line below returns the right ARN
152
+ # (`arn:aws:iam::<acct>:user/<name>`).
153
+ #
154
+ # For SSO / aws-vault / any assumed-role session this returns
155
+ # `arn:aws:sts::<acct>:assumed-role/<RoleName>/<SessionName>`, which
156
+ # Lake Formation accepts but stales on the next session refresh
157
+ # because the SessionName changes. Set PRINCIPAL_ARN to the canonical
158
+ # `arn:aws:iam::<acct>:role/<RoleName>` instead — for example:
159
+ # export PRINCIPAL_ARN="arn:aws:iam::123456789012:role/MyDevRole"
160
+ export PRINCIPAL_ARN="$(aws sts get-caller-identity --query Arn --output text)"
129
161
 
130
162
  npm install
131
163
  npx jest # 100% line + branch coverage, 95% gate
@@ -150,7 +182,7 @@ import {
150
182
  import {
151
183
  IcebergTable,
152
184
  IcebergType,
153
- } from './iceberg';
185
+ } from 'cdk-glue-iceberg-table';
154
186
 
155
187
  const db = new Database(this, 'Db', {
156
188
  databaseName: 'analytics',
@@ -204,7 +236,7 @@ import {
204
236
  IcebergSortDirection,
205
237
  IcebergTable,
206
238
  IcebergType,
207
- } from './iceberg';
239
+ } from 'cdk-glue-iceberg-table';
208
240
 
209
241
  new IcebergTable(this, 'OrdersTable', {
210
242
  database: db,
@@ -592,7 +624,7 @@ arceus/
592
624
  │ ├── iceberg-type.ts # IcebergType + struct/list/map/decimal/fixed
593
625
  │ ├── iceberg-partition-transform.ts
594
626
  │ ├── iceberg-table-properties.ts # Format/version enums + property validation
595
- │ └── index.ts # Re-exports
627
+ │ └── index.ts # Re-exports (the npm package's entry point)
596
628
  ├── test/
597
629
  │ ├── arceus-stack.test.ts
598
630
  │ ├── iceberg-evolution-stack.test.ts
@@ -601,11 +633,29 @@ arceus/
601
633
  │ ├── iceberg-table-properties.test.ts
602
634
  │ ├── iceberg-table.test.ts
603
635
  │ └── iceberg-type.test.ts
636
+ ├── e2e-consumer/ # Standalone CDK app that consumes the
637
+ │ │ # published npm package — proves a fresh
638
+ │ │ # install + import + synth works. Runs
639
+ │ │ # on every PR via the `e2e-consumer` job
640
+ │ │ # in `.github/workflows/ci.yml`.
641
+ │ ├── bin/app.ts
642
+ │ ├── lib/consumer-stack.ts # Realistic consumer (one IcebergTable)
643
+ │ └── lib/surface-reference.ts # Anchors every exported symbol so a
644
+ │ # rename in the published surface breaks CI
604
645
  ├── scripts/
605
646
  │ └── integration-test-evolution.sh # End-to-end evolution harness
647
+ ├── docs/
648
+ │ └── integ-test-setup.md # AWS-side prerequisites for the integ-test
649
+ │ # workflow (OIDC provider, IAM role, repo var)
650
+ ├── .github/workflows/
651
+ │ ├── ci.yml # Lint + test + build + pack + e2e-consumer
652
+ │ ├── publish.yml # Trusted-publish to npm on version bump
653
+ │ └── integ-test.yml # Real-AWS evolution test (gated by label /
654
+ │ # `/run-integ-test` comment)
606
655
  ├── cdk.json
607
- ├── package.json
656
+ ├── package.json # The published package's manifest
657
+ ├── tsconfig.json # Wide include (used by ESLint and dev)
658
+ ├── tsconfig.build.json # Narrow include (used by `npm publish`)
608
659
  ├── jest.config.js
609
- ├── eslint.config.js
610
- └── tsconfig.json
660
+ └── eslint.config.mjs
611
661
  ```
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "cdk-glue-iceberg-table",
3
- "version": "0.1.5",
3
+ "version": "0.1.7",
4
4
  "license": "MIT",
5
5
  "description": "CDK L2 construct for Apache Iceberg tables in the AWS Glue Data Catalog.",
6
6
  "author": "Rodrigo Carvajal",