cdk-preflight 0.0.168 → 0.0.170

package/.claude/skills/add-preflight-rule/SKILL.md

@@ -29,6 +29,7 @@ cdk-preflight のルール追加パイプライン。AGENTS.md の設計原則
 4. **ローカルゲート**: まず `npx ts-node --transpile-only --project test/tsconfig.json scripts/rule-check.ts check <service|rule-id>...` を回す。`rules/` を直接読んで 1 エンジンに全ルールを載せ、fail が自分のルールで鳴るか / pass が全ルール無音か / どちらも組み込みエンジンに止められないかを返す（`bundle-rules` も meta.yaml の evidence も要らないので、実機ゲート前の直しはここで回す。80 本で数秒）。全部 `ok` になってから `npx projen bundle-rules && npx jest test/rules test/structure.test.ts`（`test/rules` は `rules.test.ts` と、ルール表を回す `rules.shard*.test.ts` の両方に当たる）（`bundle-rules` は `src/rules.generated.ts` / `docs/rules.md` に加えて **README のルール数バッジと対応リソースタイプ一覧も書き換える**。手で直さない。古いままなら `structure.test.ts` が落ちる）。**`rules/<service>/` を新設しても `npx projen` は要らない** — `.github/workflows/monthly-verify.yml` のサービス行列は実行時に `scripts/plan-services.py` が `rules/` を読んで組むので workflow ファイル自体は変わらない（#247 で実行時計算へ移行済み。2026-09-26 に `rules/verifiedpermissions/` を新設した PR #286 / #287 でも self-mutation は skipping のままだった）。2026-09-13 に Athena 追加で踏んだのは移行前の話。`structure.test.ts` は手順 3 の境界値を機械で見る（rego のしきい値ごとに fail へ「限界に最も近い違反値」、pass へ「限界そのもの」が現れるか）。しきい値の検出が意味を持たないルールだけ `rules/_boundary-exceptions.txt` に理由付きで逃がす。**jest は `-t` で対象を絞る**。フルスイートは PR 直前の 1 回だけでよく、実測では 401 回中 73 回がフル実行で合計 5.8 時間を溶かしている。
 5. **実機再現ゲート**: `bash bench/verify-rule.sh <rule-id>`（要 AWS 認証）。観測したエラーメッセージと日付を `meta.yaml#repro.evidence` に記録。
    - **evidence には実際にデプロイした値を書く**（`bench 2026-09-13 us-east-1: 19-char Value -> "...at least 20 characters" (ROLLBACK_COMPLETE); pass 20-char -> CREATE_COMPLETE`）。境界ちょうどのフィクスチャと組で、限界の位置そのものが meta.yaml から読める
+   - **evidence の拒否文から、制約を名指ししている部分を `repro.expect` に写す**（`expect: "must not be longer than 256 characters"`）。1 行のダブルクォート文字列で、中に `"` と `\` を入れない（月次ランナーは YAML パーサ無しに sed で読む）。evidence に逐語で含まれていないと `bundle-rules` が落ちる。月次はこの文面で照合し、別の理由で倒れたら INCONCLUSIVE にする（#264）。**2026-09-27 以降に足す real-deploy ルールでは必須**。サービスが汎用文しか返さない場合（`GeneralServiceException` など）は、その汎用文を写す
    - fail テンプレートがデプロイに**成功**したら、それはドキュメント側の誤り（BROKEN-EXPECTATION）。ルールを削除し、証拠を issue に残して終了する。CloudFront では明文化された制約 9 件中 3 件がこれだった（2026-09-02）
    - **予想と違う理由**で失敗した場合（他アカウントの ARN、ドメイン所有権の検証など）は証拠にならない。サービスエラーが対象の制約そのものを名指しするまでテンプレートを作り直すか、除去できない交絡は `evidence` に明記する
    - `doc-only` は「再現に安価に作れないリソース（検証済み ACM 証明書、所有ドメイン等）が要る」場合に限る最終手段であって、まだ試していない制約への近道ではない。詳細は AGENTS.md の "A doc sentence is a hypothesis, not evidence" に従う

package/.jsii

@@ -9600,6 +9600,6 @@
       "symbolId": "src/index:PreflightOptions"
     }
   },
-  "version": "0.0.168",
-  "fingerprint": "G3aMkBNCkKzel6i+CuxZmGgbZ9hnfizvksJTGdXg26M="
+  "version": "0.0.170",
+  "fingerprint": "ncgIOw3ylx2G0+dxkUBLet0Q247HUVj+5EZi2gnZwws="
 }

package/AGENTS.md

@@ -163,7 +163,7 @@ If you are a subagent running one of these phases, do not spawn further agents.
    - A helper several rules of one service need (parsing an opaque DSL, enumerating nested scopes) goes in `rules/_lib/<service>.rego` with a `_pf_<service>lib_` prefix (`rules/_lib/sfn.rego` is the reference: ASL from `DefinitionString` *or* the L1 `Definition` object, nested Parallel/Map scopes to depth 3, effective QueryLanguage). Rules never reference another rule's helpers — `exclude` can unload any rule, a lib is always loaded.
    - **Cheap screen for definition-level constraints**: some services expose their create-time validator as a free API (`aws stepfunctions validate-state-machine-definition --type STANDARD|EXPRESS` is the same validator CreateStateMachine runs). Use it to triage doc hypotheses before spending a CloudFormation deploy on each — 100 Step Functions hypotheses took minutes (2026-09-05). The real-deploy gate stays.
 3. `npx ts-node --transpile-only --project test/tsconfig.json scripts/rule-check.ts check <service|rule-id>` while iterating: it reads `rules/` straight from disk into a single engine (no bundle, no meta validation, no jest) and reports, per rule, whether the fail template fires its own rule, the pass template is silent for every rule, and neither trips a built-in ERROR/FATAL. Then `npx projen bundle-rules` and `npx jest test/rules test/structure.test.ts` (the `test/rules` pattern picks up both `rules.test.ts` and the `rules.shard*.test.ts` files that carry the per-rule table) — the duplication guard and fixture checks run there for real. Prefer `jest -t` while iterating; the full suite is ~11,400 tests / ~2.5 min on 8 cores (measured 2026-09-14), so keep it for the pre-PR run.
-4. **Real-deploy gate**: `bash bench/verify-rule.sh <rule-id>` deploys the fail template (expects CREATE to fail; records the service error message) and, where cheap, the pass template (expects success, then deletes). Paste the observed error into `meta.yaml#repro.evidence` with the date. Only `doc-only` rules may skip this, with justification.
+4. **Real-deploy gate**: `bash bench/verify-rule.sh <rule-id>` deploys the fail template (expects CREATE to fail; records the service error message) and, where cheap, the pass template (expects success, then deletes). Paste the observed error into `meta.yaml#repro.evidence` with the date, then copy the part of it that names the constraint into `meta.yaml#repro.expect` — one line, `expect: "<fragment>"`, no `"` or `\` inside (the monthly runner reads it with `sed`), verbatim from the evidence (`bundle-rules` checks). Every monthly run matches the fail stack's reason against it, so a stack that falls over for anything else is INCONCLUSIVE rather than verified. Required for `real-deploy` rules added on or after 2026-09-27; when the service only ever says something generic (`GeneralServiceException`), quote that. Only `doc-only` rules may skip this, with justification.
 5. Update nothing else by hand, and **do not commit the generated artifacts** — a rule PR touches `rules/**` and
    nothing else. `src/rules.generated.ts` is gitignored and written by `npx projen bundle-rules`, which `preCompile`
    spawns on every build. `docs/rules.md` and the parts of `README.md` between the `<!-- supported-resources:start -->`
@@ -201,7 +201,8 @@ If you are a subagent running one of these phases, do not spawn further agents.
 - **`json.unmarshal` and `json.is_valid` exist** (measured 2026-09-03, 1.7.0-beta). Opaque JSON-in-a-string properties (CloudWatch `DashboardBody`) can be parsed and validated structurally in Rego — this is the lens-2 workhorse. Reference implementations: `pf-cloudwatch-dashboard-*`.
 - **The Rego regex builtins exist** (`regex.match`, `regex.is_valid`, `regex.find_n`, `regex.replace`) — but the engine is Rust, so its regex flavour is **not** the one AWS's published patterns assume. Two traps, both measured 2026-09-04, both of which shipped broken rules before they were caught:
   - *Character classes are Unicode-aware.* `\w`, `\d` and `\s` match non-ASCII — `regex.match("[\\w]+", "テスト用")` is **true**. AWS API patterns like Cognito's `[\w\s+=,.@-]+` are ASCII-semantics, so copying one verbatim yields a rule that never fires. Write explicit ranges (`^[A-Za-z0-9_ \t+=,.@-]+$`).
-  - *No lookahead/lookbehind.* ELBv2 publishes `(?!^-)(?!.*-$)^[A-Za-z0-9-]+$` for `targetGroupName`; that pattern does not compile here, and `regex.match` then returns false for **every** input — the rule fires on all values, which is worse than not firing. Rewrite without lookaround: `^[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?$`.
+  - *No lookahead/lookbehind.* ELBv2 publishes `(?!^-)(?!.*-$)^[A-Za-z0-9-]+$` for `targetGroupName`; that pattern does not compile here, and `regex.match` is then undefined for **every** input — so the usual `not regex.match(...)` shape is `not undefined` = true and the rule fires on all values, which is worse than not firing. Rewrite without lookaround: `^[A-Za-z0-9]([A-Za-z0-9-]*[A-Za-z0-9])?$`.
+- **A Unicode class under a large bounded repeat is rejected by the regex engine, and the rule then breaks whichever way it was written** (measured 2026-09-27). AWS publishes Grafana's `workspaceDescription` as `^[\p{L}\p{Z}\p{N}\p{P}]{0,2048}$`; handed to `regex.match` verbatim it does not compile. The bound itself is not the story — `[a-z]{0,100000}` compiles, while four `\p` classes already break at `{0,500}` and a single `\p{L}` at `{0,2048}` — so what blows up is the size of the compiled pattern (class width × repeat count), not a repeat-count cap; do not port Go's "repeat count over 1000" rule of thumb to this engine. On a rejected pattern `regex.match` is **undefined, not false** (binding it kills the body; nothing is thrown and the rest of the pack keeps firing), exactly like the lookaround trap above — so which way the rule breaks depends on how it is written: `not regex.match(p, v)`, the shape these charset checks almost always take, is `not undefined` = true and fires on **every** value (`pass: <id> fired`), while a positive `regex.match(p, v)` guard goes **silent** (`fail: own rule silent`). `regex.is_valid(p) == false` catches both and is the one-line self-check. Split the length off and leave it to the schema or `count()`: `pf-grafana-ws-description-charset` matches the charset with `*` and drops the 2048.
 - **The absence sentinel must not be a string.** The `object.get(props, "<Key>", "__pf_absent")` idiom is only safe when the next step is `== "__pf_absent"`. If the next step is `is_string(v)` followed by a format check, an *absent* key yields the sentinel string and the rule fires on every resource that omits the property. Use `null` as the default in that case — `is_string(null)` is false, so the rule skips. (This produced three false-positive rules in the 2026-09-04 batch, visible only because other rules' pass fixtures went red.)
 - **`not <undefined>` is true, so a helper that declines makes the rule fire on everything.** Rego's negation succeeds when its operand is undefined, so `not a in _helper(x)` passes whenever `_helper(x)` is undefined — not just when `a` is missing from it. Helpers guard their input (`is_array`, `is_object`, an unresolved `Fn::If` marker) precisely so they decline on the values a rule should skip, and writing the negation inline turns that guard inside out into a false positive on every such resource. Bind first — `defined := _helper(x)` on its own line, then `not a in defined` — so the body goes undefined and the rule stays silent when the helper declines. **Neither fixture catches this**: both templates carry a value the helper accepts, so only a false-positive sweep over unrelated templates finds it. Measured twice on 2026-09-25: the OpenSearch Serverless policy rules fired on the `Fn::Join`+`Ref` policies CDK ordinarily emits (#268), and the Network Firewall custom-action rules fired on every stateless rule group (#280).
 - **Two bare `some x` declarations in one rule body take the whole pack down** (measured 2026-09-09, 1.7.0-beta): the engine defines the second one's local slot once per binding of the first, so as soon as the outer iteration yields two values the custom package dies with `duplicated definition of local variable <name>` — every rule stops firing, for every stack, and `exclude` cannot work around it because the package fails as a whole. One filter entry passed, two did not (issue #150). Write the second iteration as `some k, v in coll` (guarded by `is_array` / `is_object`, since `some .. in` on a scalar is itself a hard error); converting either of the two is enough. `test/structure.test.ts` fails the build on any body with two bare `some`s. Reference: `pf-wafv2-logging-filter-condition`.
@@ -229,6 +230,7 @@ If you are a subagent running one of these phases, do not spawn further agents.
 - **`some` cannot re-declare a variable that is already bound in the same body or an enclosing one** ("var `name` used before definition below"). Iterating the same tuple set twice needs fresh names plus equality checks (`some [nm, ii, m, l2, c2] in set; nm == name; ii == i`), and a comprehension inside a body must not reuse the body's variable names.
 - **`sprintf` with the wrong arity silently disables the rule**, like a bare `%`: `sprintf("%d %s", [n], extra)` never evaluates and nothing warns (measured 2026-09-05). Keep every argument inside the array.
 - **`%q` is not a supported `sprintf` verb** (measured 2026-09-24): the policy compiles, but the body never evaluates and no diagnostic fires — the same silent death as a bare `%`. Use `'%s'` (quote it yourself) or `%v`.
+- **A `sprintf` format string with fewer verbs than array elements dies the same way, and a generator is what produces it** (measured 2026-09-27): run the format string through a `%`-style templater on the way into the `.rego` file and `%v` comes out as `%%v` — a literal percent followed by a `v`, so the format carries zero verbs while the array still carries one element, the body never evaluates and nothing warns (`sprintf("a %%v b", ["x"])` is silent; `sprintf("a %%v b", [])` yields `a %v b`). This is not the wrong-arity trap above — there the `sprintf` *call* is malformed; here the call is well-formed and only the verb count is off. Keep rego in raw string literals and pass it through no `%` formatting at all: in #290 one slice built its message with Python's `%` operator and silenced a rule, and the slice that kept the rego raw was 9/9 `ok` on the first run. `rule-check`'s `fail: own rule silent` is the only thing that shows it.
 - **`meta.fixtureRegion` moves a rule's fixture evaluation off the harness default (us-east-1)**, in both harnesses (`test/rules.test.ts` and `scripts/rule-check.ts`; it wins over `PF_REGION`). Needed for rules about the deploy region itself (`pf-wafv2-scope-region`: CLOUDFRONT scope only in us-east-1) whose fail template cannot fire where the harness runs.
 - **WAFv2 names are unique per scope and region, so parallel bench runs collide on fixture names.** The WAFv2 generators suffix every `Name` with the rule id (`uniq()` in the scratch `gen-lib.js`); do the same for any service whose entity names are account-unique.
 - **The CLI's service model can lag the API** (measured 2026-09-05: `aws wafv2` had no `Monetize` action and no `PreParseTextTransformations`). Fields the CLI rejects with ParamValidation need a stack for triage; everything else is cheaper through `create-*` calls whose rejections create nothing (WAF: ~290 calls, 13 stacks for the whole survey). `aws wafv2 check-capacity` is the oracle for the WCU estimate in `pf-wafv2-capacity`.
@@ -246,6 +248,7 @@ If you are a subagent running one of these phases, do not spawn further agents.
 - **Fixture name tagging must skip reference properties.** The generator that suffixes fixture names with the rule id has to key off the resource type: `ACLName` is the name on `AWS::MemoryDB::ACL` but a *reference* on `AWS::MemoryDB::Cluster`, and tagging it made every MemoryDB pass template fail with "ACL open-access-port not found" (measured 2026-09-06). Same for `ParameterGroupName` / `CacheParameterGroupName`.
 - **`bench/verify-rule.sh` polls for 60 minutes** (15 → 30 on 2026-09-06, → 60 on 2026-09-13): ElastiCache and MemoryDB clusters take 10-25 minutes to create and about as long to roll back, and an MSK Provisioned cluster took 31m34s to reach CREATE_COMPLETE at its minimum size; a shorter poll reported TIMEOUT / INCONCLUSIVE on rules whose constraint had already fired.
 - **A non-zero `aws` exit is not evidence of anything** (measured 2026-09-22, four parallel `verify-rule.sh` in us-east-1). `describe-stacks … 2>/dev/null || echo GONE` collapsed throttling, blips and expired credentials into `GONE`; `cleanup()` then deleted two `pf-servicediscovery-*` stacks mid-create, and in one of them `reason_of` captured the real failure a moment later — proof from the run's own log that the stack still existed. The same collapse on `describe-stack-events` is worse than losing a run: an unreadable reason leaves `FTYPE`/`REASON` empty, the scaffolding guard silently no-ops, and a VPC-quota rollback reports `OK: verified`. Only `ValidationError … does not exist` means gone. Everything else is retried inside the poll budget and then reported under its own reason (`could not read the … status` / `… events`), never as `GONE` and never as a dirty fixture; a `LEFTOVER` line likewise needs a confirmed-present stack, not just a `wait` that failed. `bench/verify-rule.test.sh` pins all of it with a stubbed `aws` on `PATH` — 13 of its assertions fail against the pre-fix script.
+- **A fail stack falling over is not evidence that the constraint fired** (measured 2026-09-25, #264). `verify-rule.sh` verified any rollback that did not look like a quota, and in the cloudformation monthly run only 9 of the 21 OKs named their constraint. Three holes. `reason_of` dropped the stack's own rows with `ResourceType!='AWS::CloudFormation::Stack'`, which also dropped a *nested* stack's real failure and picked up a sibling's collateral `Resource creation cancelled` instead; the stack's own rows are the ones whose `LogicalResourceId` is the stack name. A failure that belongs to no resource (an Output's Export) puts its reason only on the stack's `ROLLBACK_IN_PROGRESS` row under the default rollback. And a synchronous `ValidationError` from `create-stack` — the strongest evidence there is, with no scaffolding to collapse — was always INCONCLUSIVE, which pinned the cloudformation monthly at `overall=4`. Now an unreadable reason, a collateral cancellation or the `The following resource(s) failed to create` summary is INCONCLUSIVE, and with `repro.expect` set the reason, or the synchronous rejection, has to contain it. Replayed over the dev account's 90-day event history (3,110 stacks, 2026-09-26), the new reader changes the reason of those 5 cloudformation rules and of no other rule. The event reads also moved to `--output json`: with `--output text` the CLI applies `--query` page by page (100 events a page), so on a stack of more than 100 events `[-1]` answered once per page, and "no reason on either page" came back as two `None` lines, which is not `None` (measured 2026-09-27: `length(StackEvents)` is `100` and `16` in text, `116` in json).
 - **Probing the raw engine with custom rules**: the rule set must be named `violation` (`data.<pkg>.violation` is looked up; a `diagnostics` set fails with "not a valid rule path"), and hand-built diagnostic objects need lowercase severities (`error`, not `ERROR`/`CUSTOM` — those are display labels added by the wrapper). Pack rules never see this because `make_diag_full` handles the shape.
 - **`as` is a reserved keyword in the engine's Rego build** (measured 2026-09-06): `acts := object.get(s, "Action", null)` is fine but `as := ...` fails to parse ("unexpected keyword `as`") and, like every parse error, takes the whole custom package down — every rule stops firing, and the jest failure names an unrelated rule.
 - **A Managed Flink application deploys with no `ApplicationConfiguration` at all** (measured 2026-09-06), which is what makes `AWS::KinesisAnalyticsV2::Application` fixtures cheap: no S3 bucket, no JAR, no Glue database, just an IAM role the service can assume. Adding a "minimal" code stanza is worse than none — CreateApplication really fetches the S3 object (`We are unable to get the specified fileKey`) and rejects `PLAINTEXT` for a Flink runtime.

package/lib/index.js

@@ -17,7 +17,7 @@ const rules_generated_1 = require("./rules.generated");
  * Preflight.apply(app);
  */
 class Preflight {
-    static [JSII_RTTI_SYMBOL_1] = { fqn: "cdk-preflight.Preflight", version: "0.0.168" };
+    static [JSII_RTTI_SYMBOL_1] = { fqn: "cdk-preflight.Preflight", version: "0.0.170" };
     /**
      * Register the cdk-preflight rules on an App or Stage.
      */

package/package.json

@@ -89,7 +89,7 @@
   "publishConfig": {
     "access": "public"
   },
-  "version": "0.0.168",
+  "version": "0.0.170",
   "jest": {
     "coverageProvider": "v8",
     "coveragePathIgnorePatterns": [