@davidvornholt/standards 0.23.0 → 0.24.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -2
- package/package.json +8 -1
- package/src/cli.ts +14 -83
- package/src/contained-path.ts +59 -0
- package/src/creds-dest.ts +4 -3
- package/src/creds-login-github-manifest.ts +1 -0
- package/src/creds-sops-structure.ts +0 -53
- package/src/creds-sops-value.ts +3 -2
- package/src/creds-sops.ts +4 -4
- package/src/creds-target.ts +3 -3
- package/src/json-file.ts +17 -0
- package/src/structure-check.ts +25 -21
- package/src/structure-readme.ts +85 -0
- package/src/structure-secret-document.ts +170 -0
- package/src/structure-secrets.ts +131 -0
- package/src/structure-sops-envelope.ts +123 -0
- package/src/sync-policy.ts +88 -0
package/README.md
CHANGED
|
@@ -35,11 +35,11 @@ standards poller --print-units --config <path>
|
|
|
35
35
|
|
|
36
36
|
`github --check` compares the live GitHub repository (including the canonical squash-only repository settings, rulesets, and declared labels) against the merged declaration in `.github/settings.json` + `.github/settings.local.json`; `github --apply` converges that declared state. The canonical declaration enables squash while disabling merge commits and rebases, and its default-branch ruleset independently allows only squash. Undeclared live labels remain untouched, while undeclared repository-owned rulesets are removed. `check` runs the same comparison whenever `.github/settings.json` is present, and fails closed: declared state the token cannot see fails the gate with a message naming the token fix instead of passing unverified. The canonical workflow sets `STANDARDS_SKIP_GITHUB_CHECK=true` only in its unprivileged quality step because its `check` aggregator job runs the live comparison with its own workflow token; this is a workflow-internal integration seam, not a general way to disable the local gate. The CLI prints an explicit diagnostic whenever that exact value skips either `check`'s live phase or `github --check`; `github --apply` is never skipped. `--apply` re-diffs the PATCH response, so a setting GitHub accepts with HTTP 200 but silently ignores (plan-unavailable features) is a reported failure, not a false success. A repository whose GitHub plan cannot enforce rulesets can declare that in the seam (see `rulesetEnforcement` below); both commands then skip rulesets and plan-gated repository settings, converge the rest, and print an unprotected-branch notice on every run.
|
|
37
37
|
|
|
38
|
-
`structure` enforces the canonical monorepo structure contract: workspace scripts (`check-types`, `lint`, `lint:fix`, `test`), root gate scripts, filtered Turbo convenience aliases, internal `0.0.0`/`workspace:*` versioning, package `exports`, shared tsconfig inheritance, and browser a11y wiring. Only a workspace containing an explicit `*.a11y.ts` suite must provide a `test:a11y` script and direct `@axe-core/playwright` and `@playwright/test` dependencies. `check` includes `structure`, so these rules gate every consumer PR.
|
|
38
|
+
`structure` enforces the canonical monorepo structure contract: workspace scripts (`check-types`, `lint`, `lint:fix`, `test`), root gate scripts, filtered Turbo convenience aliases, internal `0.0.0`/`workspace:*` versioning, package `exports`, shared tsconfig inheritance, and browser a11y wiring. Only a workspace containing an explicit `*.a11y.ts` suite must provide a `test:a11y` script and direct `@axe-core/playwright` and `@playwright/test` dependencies. Every repo-owned workspace must carry a non-empty `README.md` as the prerequisite for documenting its configuration and secrets; the gate checks presence, not content completeness. Canonical workspaces synced from the standards template (the `apps/*` or `packages/*` entries in `sync-standards.json`) are exempt under the consumer profile because a consumer cannot fix them locally, and an unreadable or symlinked `sync-standards.json` fails the gate rather than guessing which workspaces are repo-owned. `structure` also validates the CI secrets contract without decrypting anything, using the plaintext key structure SOPS leaves in place: `secrets/ci.yaml` must be a contained regular file with complete SOPS metadata, every business leaf value must have a complete `ENC[AES256_GCM,...]` envelope (a rejected leaf is reported by key path only, never by value), and the repo-owned `secrets/ci.example.yaml` must mirror the structural business-key paths in both directions with plaintext placeholders. `ci.ntfy_topic_url` is always required for `Notify pause`. `ci.broker_app.app_id` and `ci.broker_app.private_key` are required for the weekly `Standards sync` workflow unless the validated `sync-standards.local.json` policy sets `"autoSync": false`. Required workflow values must each be one encrypted scalar because the resolver requires a string. The secrets files, workspace READMEs, and sync manifest cannot be supplied through symlinks. `check` includes `structure`, so these rules gate every consumer PR.
|
|
39
39
|
|
|
40
40
|
`poller` runs one tick of the fix poller: host-level automation that turns maintainer-approved GitHub issues into verified draft PRs and maintainer-approved draft PRs into completed review-fix cycles, using headless Codex runs. All workflow state lives in GitHub labels, comments, and draft PRs (the labels are canonical declared settings — see `github --apply`), so ticks are stateless, re-runnable, and safe to schedule from any host. A tick claims work by label (`fix-in-progress` / `review-in-progress`), releases claims stale for longer than `staleClaimHours`, and fails loudly — a red tick means jobs stalled, never silence. A separate acknowledgement tick validates pending approvals and posts one deduplicated queued message per approval generation; it never claims or starts work, so it can keep giving maintainers prompt feedback while the worker is occupied. When a worker claims a request, it posts a concise start message. The exact approval and distributed claim records remain in hidden comment metadata, and losing claim contenders remove their duplicate comments. Trust is verified per event, not per configuration: the approval labels (`approved-for-fix`, `approved-for-review`) only count when the applying user holds the `admin` or `maintain` role, bind to the exact approved issue revision or PR head, and are revalidated before publication; only admin/maintain comments are treated as answers to agent questions. Approved Codex runs inherit the poller's GitHub token and use authenticated `gh` for relevant reads and comments; review runs also maintain the PR review ledger, apply authorized title or body edits, post fix threads, and file deferred findings directly. The run works in a throwaway git worktree, grants Codex write access to that worktree and its exact Git common directory so commits can update the linked repository metadata, and hands results back through a structured outcome file. The poller then verifies the approved head, commits, and protected paths before pushing or publishing; it replies to and resolves agent-posted fix threads only after the reviewed head is pushed, and it alone changes protocol labels, posts the final report, and flips a reviewed draft ready, while required CI remains the quality gate. If publishing a sealed fix or review result fails, the poller releases only its owned in-progress claim and keeps the approval and sealed output, so the next worker tick retries publication without another Codex run. Sealed review plans from the earlier protocol are rejected and their output branches are ignored, so the review reruns on a fresh protocol-versioned branch rather than silently dropping legacy deferred work. Host mutation is declarative-only: `--print-units` emits four correctly escaped systemd units for the polling host's infrastructure repository to adapt and deploy: the worker service and timer plus an independent lightweight acknowledgement service and timer. That repository owns the service identity, writable HOME, PATH, authenticated `gh`, lingering, and deployment. The worker service's tick budget (`TimeoutStartSec`) is derived from `maxJobsPerTick` × `runTimeoutMinutes` plus overhead, so re-render and redeploy the units after changing either value.
|
|
41
41
|
|
|
42
|
-
`creds` is the credential broker: it mints scoped Cloudflare tokens, rotates and revokes those tokens, and places GitHub App credentials into SOPS-encrypted targets — secret values never touch stdout, argv, or an agent's context; commands print only metadata (token name, scopes, expiry, destination). Cloudflare reconciliation has exactly two sources of truth: the plaintext key structure of the SOPS files and the provider token list under the deterministic naming scheme `standards/<owner>/<repo>/<sops-target>/<dotted.key>`; there is no separate credential manifest. `plan` and `apply` reconcile Cloudflare only: each run uses the stored bootstrap credential to verify its current provider token ID, excludes that ID from reconciliation regardless of its current name, and aborts if the identity cannot be correlated with the complete account token list; a brokered token whose SOPS key was deleted is revoked, while one nearing expiry (30 days) is renewed by creating a fresh-expiry replacement with the live token's policies, durably writing and verifying the new value, and only then revoking the old token; secrets the broker did not mint are never touched. Both also report what reconciliation will never touch, in two separately labelled and separately counted blocks that never block: unmanaged tokens, meaning every unexpired token in the account whose name falls outside the reserved `standards/` namespace — with one exception, this machine's own bootstrap credential, which every run removes by verified ID before anything is classified — and tokens brokered to another repository, each printed with the repository whose `apply` owns it. An expired token is omitted from the unmanaged listing because it grants nothing. Neither block is idle information: nothing here renews or revokes those tokens, and a forgotten one — the original half of a hand-made credential that was replaced rather than retired, or a brokered token whose repository was renamed away — stays valid indefinitely, so a listed token is normally printed with the exact command that retires it, carrying `--account` and, for a token brokered elsewhere, `--force` and the one condition that justifies it. Two rows are exceptions. A token named `standards-broker` holds the reserved name for a machine's bootstrap credential, so it is either another machine's or a superseded one on this machine; `revoke` refuses it by name, so the row names the Cloudflare dashboard rather than a command that would always fail. The other exception looks like an ordinary brokered-elsewhere row, and its command fails. `plan` classifies by exact token name, so a token whose brokered repository differs from this checkout's origin only in capitalisation lands in the brokered-elsewhere block with a `--force` command; `revoke` compares repository names case-insensitively, reads the same token as this checkout's own, and refuses. Row and block header both read that token as foreign — the header labels the block tokens brokered to another repository, and the row explains the `--force` by the owning repository having been renamed, transferred, or deleted. Neither holds there: GitHub names are case-insensitive, so the owning repository is this one and it is still alive. Re-point the origin remote at the capitalisation the token carries, or retire the token in the Cloudflare dashboard. A name that claims this repository's brokered namespace without being one the broker mints is a finding instead, and findings abort reconciliation before any mutation, because reconciliation cannot own that token; plain `revoke --token-id` takes it once you decide it should go. `revoke --token-id <id>` deletes exactly one Cloudflare token, named by its 32-character hexadecimal ID, with `--account <id>` required whenever more than one account is configured; it records no ownership, so the power is momentary rather than standing and a human names the target every time. It refuses an ID the account does not hold, the bootstrap credential this machine has stored (matched by verified ID, so a rename cannot disguise it), and any other token named `standards-broker`, the reserved name for a machine's bootstrap (matched by name, because that one is another machine's bootstrap or a superseded one, retired in the Cloudflare dashboard where you can confirm whose it is — the broker cannot re-mint its own root credential). A token brokered to this repository is refused with the remedy that works here: delete its SOPS key in this repository and run `apply`, which revokes the token and keeps the secret in step. A token whose brokered name differs from this checkout's origin only in capitalisation is refused too, and without the `apply` remedy: reconciliation matches the name exactly, so from here it would neither renew nor revoke the token. Its remedies are re-pointing the origin remote at the capitalisation the token carries, which brings it back under reconciliation, or retiring it in the Cloudflare dashboard. A token brokered to a repository that is genuinely a different one is refused with that repository named, because the same `apply` remedy runs there. `--force` widens exactly that last refusal, and nothing else: it lets `revoke` delete a token whose brokered repository is not this checkout's. It verifies nothing about the state of that repository, so it is justified only when nothing reconciles the token any more — the owning repository was renamed, transferred, or deleted, and no checkout resolves to its name. `revoke` decides ownership from the origin remote and compares it case-insensitively, as GitHub names are; `plan` classifies by exact name instead, which is why the two disagree about a case-only difference. When the origin remote resolves to no GitHub repository, every brokered token is refused, `--force` included, because the check it widens is the one that could not be run. `--force` never bypasses either bootstrap guard, the case-only refusal, or that unresolvable-origin refusal. Without a manifest there is no desired-policy record, so live policy drift is not detected, and a repository rename or transfer changes the deterministic namespace and requires the visible old token to be re-minted under the new name and revoked. `add cloudflare` mints an account-owned API token (default TTL 90 days) scoped to the named permission groups (`permissions` lists the valid names) and writes it to `--dest <target>:<dotted.key>`, where `<target>` resolves like the canonical `just secrets` targets (`ci` → `secrets/ci.yaml`, `prod-1` → `infra/hosts/prod-1/secrets.yaml`). `--zone <zone-id>[,<zone-id>...]` adds a zone resource so zone-scoped groups such as DNS Write can be minted. Zones are named by their 32-character hexadecimal ID, shown on the zone's dashboard overview and not a secret; resolving a domain name instead would need Zone Read on the bootstrap token, which holds Account API Tokens / Edit and nothing else. With zones named, the selection is split by what each group can target: every group Cloudflare reports as zone-scoped forms one policy holding all named zones, and groups that can only target an account keep their own account policy on the same token, so one credential can hold DNS access next to account-wide access. A group Cloudflare reports as both takes the zone policy, which is the narrower reading of `--zone`. A group matching neither resource is named and rejected, `--zone` requires at least one zone-scoped group, and `--zone` cannot be combined with `--bucket`, which is a separate credential. The mint summary prints the resources the token reaches, so an account policy alongside the zones is visible rather than implied. `--bucket <name>` switches the policy resource to a single R2 bucket; `--jurisdiction default|eu` selects the provider jurisdiction and defaults to `default`, producing `com.cloudflare.edge.r2.bucket.<account_id>_default_<bucket>` or `com.cloudflare.edge.r2.bucket.<account_id>_eu_<bucket>`. FedRAMP jurisdictions are not supported. Bucket destinations accept only bucket-scoped groups such as Workers R2 Storage Bucket Item Read/Write, and account-scoped groups are rejected. `--s3` requires `--bucket` and stores the S3-compatible credential pair Cloudflare derives from an API token — access key ID is the token ID, secret access key is the SHA-256 hex digest of the token value — as `<dotted.key>.access_key_id` and `<dotted.key>.secret_access_key` under the destination instead of the raw token value; the default endpoint is `https://<account_id>.r2.cloudflarestorage.com`, while EU uses `https://<account_id>.eu.r2.cloudflarestorage.com`. Reconciliation infers each destination's concrete write footprint from the plaintext SOPS key structure — a bearer owns the leaf at its key, while an S3 token owns both pair leaves — and aborts before mutation when managed footprints intersect. `plan` and `apply` renew and revoke S3 destinations like any other brokered token, but renewal first proves the stored access key ID matches the current provider token; an incomplete pair, unreadable value, or identity mismatch is surfaced as a finding and never mutated automatically. `add github` selects the private broker App whose owner matches the destination repository's origin owner, checks both local SOPS leaves before authenticating, proves that the exact selected App is installed on that repository, and writes only while that full App identity still holds the synchronized broker-store entry; missing, duplicate, replaced, or mismatched owners fail before SOPS changes. Durable GitHub tokens cannot be created via API (PAT and OAuth-app creation are UI-only), so workflows mint short-lived installation tokens from the App at runtime (`actions/create-github-app-token`), scoped per repository and permission. The canonical weekly sync expects `bun standards creds add github --dest ci:ci.broker_app`, then mints a current-repository token with Contents read and Pull requests write; missing credentials or minting failures stop it without a fallback. Bootstrap is once per machine and per GitHub owner: `login github` creates a private personal App, while `login github --org <org>` creates or replaces only that organization's entry; each manifest conversion records GitHub's authoritative owner, and the command opens the installation page, where the App must be bound to selected repositories rather than all repositories. GitHub App names are globally unique and limited to 34 characters, so organization defaults include owners up to 17 characters; a longer owner or a colliding replacement name requires an explicit shorter unused `--name`. The first GitHub-specific command against an existing singleton broker store authenticates as that App, resolves its registration owner, and atomically migrates the old mapping to that one owner; a failed lookup leaves the persisted bytes untouched. If the singleton was a public App installed on repositories owned by other users or organizations, those repositories stop matching after migration because owner selection no longer treats that App as shared. For each affected owner, run `login github` while signed in as that personal owner or `login github --org <owner>`, install the new private App only on the selected repositories, run `bun standards creds add github --dest <target>:<key>` in every consumer, and verify its workflows can mint tokens. Retire the old public App only after every repository owner has its own entry, every intended SOPS destination has been reprovisioned, and token minting has been verified; until then, keep its key and installation available for any destinations not yet migrated. `login cloudflare` guides the one dashboard paste Cloudflare requires for the first account token (named `standards-broker` to match the App, created from the "Create additional tokens" token template because its "Account API Tokens: Edit" permission is unavailable in the custom token builder, and holding that permission only), verifies that it is active, correlates its verified ID with the complete account token list and an active list record, and stores it; a bootstrap token named inside the minted `standards/` namespace is rejected before storage to keep its role unambiguous, while any other divergence from the recommended name only warns. `plan` and `apply` do not rotate GitHub App keys; create a replacement key on the App's settings page, atomically replace that owner's `private_key` in the protected machine-global broker store while preserving its `0600` mode, then update every intended SOPS destination, verify workflows can mint installation tokens, and revoke the old key manually. Root credentials live in the machine-global broker store (see Configuration); SOPS writes go through `sops edit` with a non-interactive editor, preserving comments and reusing the file's data key.
|
|
42
|
+
`creds` is the credential broker: it mints scoped Cloudflare tokens, rotates and revokes those tokens, and places GitHub App credentials into SOPS-encrypted targets — secret values never touch stdout, argv, or an agent's context; commands print only metadata (token name, scopes, expiry, destination). Cloudflare reconciliation has exactly two sources of truth: the plaintext key structure of the SOPS files and the provider token list under the deterministic naming scheme `standards/<owner>/<repo>/<sops-target>/<dotted.key>`; there is no separate credential manifest. `plan` and `apply` reconcile Cloudflare only: each run uses the stored bootstrap credential to verify its current provider token ID, excludes that ID from reconciliation regardless of its current name, and aborts if the identity cannot be correlated with the complete account token list; a brokered token whose SOPS key was deleted is revoked, while one nearing expiry (30 days) is renewed by creating a fresh-expiry replacement with the live token's policies, durably writing and verifying the new value, and only then revoking the old token; secrets the broker did not mint are never touched. Both also report what reconciliation will never touch, in two separately labelled and separately counted blocks that never block: unmanaged tokens, meaning every unexpired token in the account whose name falls outside the reserved `standards/` namespace — with one exception, this machine's own bootstrap credential, which every run removes by verified ID before anything is classified — and tokens brokered to another repository, each printed with the repository whose `apply` owns it. An expired token is omitted from the unmanaged listing because it grants nothing. Neither block is idle information: nothing here renews or revokes those tokens, and a forgotten one — the original half of a hand-made credential that was replaced rather than retired, or a brokered token whose repository was renamed away — stays valid indefinitely, so a listed token is normally printed with the exact command that retires it, carrying `--account` and, for a token brokered elsewhere, `--force` and the one condition that justifies it. Two rows are exceptions. A token named `standards-broker` holds the reserved name for a machine's bootstrap credential, so it is either another machine's or a superseded one on this machine; `revoke` refuses it by name, so the row names the Cloudflare dashboard rather than a command that would always fail. The other exception looks like an ordinary brokered-elsewhere row, and its command fails. `plan` classifies by exact token name, so a token whose brokered repository differs from this checkout's origin only in capitalisation lands in the brokered-elsewhere block with a `--force` command; `revoke` compares repository names case-insensitively, reads the same token as this checkout's own, and refuses. Row and block header both read that token as foreign — the header labels the block tokens brokered to another repository, and the row explains the `--force` by the owning repository having been renamed, transferred, or deleted. Neither holds there: GitHub names are case-insensitive, so the owning repository is this one and it is still alive. Re-point the origin remote at the capitalisation the token carries, or retire the token in the Cloudflare dashboard. A name that claims this repository's brokered namespace without being one the broker mints is a finding instead, and findings abort reconciliation before any mutation, because reconciliation cannot own that token; plain `revoke --token-id` takes it once you decide it should go. `revoke --token-id <id>` deletes exactly one Cloudflare token, named by its 32-character hexadecimal ID, with `--account <id>` required whenever more than one account is configured; it records no ownership, so the power is momentary rather than standing and a human names the target every time. It refuses an ID the account does not hold, the bootstrap credential this machine has stored (matched by verified ID, so a rename cannot disguise it), and any other token named `standards-broker`, the reserved name for a machine's bootstrap (matched by name, because that one is another machine's bootstrap or a superseded one, retired in the Cloudflare dashboard where you can confirm whose it is — the broker cannot re-mint its own root credential). A token brokered to this repository is refused with the remedy that works here: delete its SOPS key in this repository and run `apply`, which revokes the token and keeps the secret in step. A token whose brokered name differs from this checkout's origin only in capitalisation is refused too, and without the `apply` remedy: reconciliation matches the name exactly, so from here it would neither renew nor revoke the token. Its remedies are re-pointing the origin remote at the capitalisation the token carries, which brings it back under reconciliation, or retiring it in the Cloudflare dashboard. A token brokered to a repository that is genuinely a different one is refused with that repository named, because the same `apply` remedy runs there. `--force` widens exactly that last refusal, and nothing else: it lets `revoke` delete a token whose brokered repository is not this checkout's. It verifies nothing about the state of that repository, so it is justified only when nothing reconciles the token any more — the owning repository was renamed, transferred, or deleted, and no checkout resolves to its name. `revoke` decides ownership from the origin remote and compares it case-insensitively, as GitHub names are; `plan` classifies by exact name instead, which is why the two disagree about a case-only difference. When the origin remote resolves to no GitHub repository, every brokered token is refused, `--force` included, because the check it widens is the one that could not be run. `--force` never bypasses either bootstrap guard, the case-only refusal, or that unresolvable-origin refusal. Without a manifest there is no desired-policy record, so live policy drift is not detected, and a repository rename or transfer changes the deterministic namespace and requires the visible old token to be re-minted under the new name and revoked. `add cloudflare` mints an account-owned API token (default TTL 90 days) scoped to the named permission groups (`permissions` lists the valid names) and writes it to `--dest <target>:<dotted.key>`, where `<target>` resolves like the canonical `just secrets` targets (`ci` → `secrets/ci.yaml`, `prod-1` → `infra/hosts/prod-1/secrets.yaml`). `--zone <zone-id>[,<zone-id>...]` adds a zone resource so zone-scoped groups such as DNS Write can be minted. Zones are named by their 32-character hexadecimal ID, shown on the zone's dashboard overview and not a secret; resolving a domain name instead would need Zone Read on the bootstrap token, which holds Account API Tokens / Edit and nothing else. With zones named, the selection is split by what each group can target: every group Cloudflare reports as zone-scoped forms one policy holding all named zones, and groups that can only target an account keep their own account policy on the same token, so one credential can hold DNS access next to account-wide access. A group Cloudflare reports as both takes the zone policy, which is the narrower reading of `--zone`. A group matching neither resource is named and rejected, `--zone` requires at least one zone-scoped group, and `--zone` cannot be combined with `--bucket`, which is a separate credential. The mint summary prints the resources the token reaches, so an account policy alongside the zones is visible rather than implied. `--bucket <name>` switches the policy resource to a single R2 bucket; `--jurisdiction default|eu` selects the provider jurisdiction and defaults to `default`, producing `com.cloudflare.edge.r2.bucket.<account_id>_default_<bucket>` or `com.cloudflare.edge.r2.bucket.<account_id>_eu_<bucket>`. FedRAMP jurisdictions are not supported. Bucket destinations accept only bucket-scoped groups such as Workers R2 Storage Bucket Item Read/Write, and account-scoped groups are rejected. `--s3` requires `--bucket` and stores the S3-compatible credential pair Cloudflare derives from an API token — access key ID is the token ID, secret access key is the SHA-256 hex digest of the token value — as `<dotted.key>.access_key_id` and `<dotted.key>.secret_access_key` under the destination instead of the raw token value; the default endpoint is `https://<account_id>.r2.cloudflarestorage.com`, while EU uses `https://<account_id>.eu.r2.cloudflarestorage.com`. Reconciliation infers each destination's concrete write footprint from the plaintext SOPS key structure — a bearer owns the leaf at its key, while an S3 token owns both pair leaves — and aborts before mutation when managed footprints intersect. `plan` and `apply` renew and revoke S3 destinations like any other brokered token, but renewal first proves the stored access key ID matches the current provider token; an incomplete pair, unreadable value, or identity mismatch is surfaced as a finding and never mutated automatically. `add github` selects the private broker App whose owner matches the destination repository's origin owner, checks both local SOPS leaves before authenticating, proves that the exact selected App is installed on that repository, and writes only while that full App identity still holds the synchronized broker-store entry; missing, duplicate, replaced, or mismatched owners fail before SOPS changes. Durable GitHub tokens cannot be created via API (PAT and OAuth-app creation are UI-only), so workflows mint short-lived installation tokens from the App at runtime (`actions/create-github-app-token`), scoped per repository and permission. The canonical weekly sync expects `bun standards creds add github --dest ci:ci.broker_app`, then mints a current-repository branch writer with Contents write and Workflows write plus a separate PR opener with Contents read and Pull requests write; both mint before sync but enter only their post-sync operations, and missing credentials, unapproved App permissions, or minting failures stop the workflow without a fallback. Existing broker Apps must add Workflows write on the App settings page and have every installation owner approve that permission before consumers adopt the workflow. Bootstrap is once per machine and per GitHub owner: `login github` creates a private personal App, while `login github --org <org>` creates or replaces only that organization's entry; each manifest conversion records GitHub's authoritative owner, and the command opens the installation page, where the App must be bound to selected repositories rather than all repositories. GitHub App names are globally unique and limited to 34 characters, so organization defaults include owners up to 17 characters; a longer owner or a colliding replacement name requires an explicit shorter unused `--name`. The first GitHub-specific command against an existing singleton broker store authenticates as that App, resolves its registration owner, and atomically migrates the old mapping to that one owner; a failed lookup leaves the persisted bytes untouched. If the singleton was a public App installed on repositories owned by other users or organizations, those repositories stop matching after migration because owner selection no longer treats that App as shared. For each affected owner, run `login github` while signed in as that personal owner or `login github --org <owner>`, install the new private App only on the selected repositories, run `bun standards creds add github --dest <target>:<key>` in every consumer, and verify its workflows can mint tokens. Retire the old public App only after every repository owner has its own entry, every intended SOPS destination has been reprovisioned, and token minting has been verified; until then, keep its key and installation available for any destinations not yet migrated. `login cloudflare` guides the one dashboard paste Cloudflare requires for the first account token (named `standards-broker` to match the App, created from the "Create additional tokens" token template because its "Account API Tokens: Edit" permission is unavailable in the custom token builder, and holding that permission only), verifies that it is active, correlates its verified ID with the complete account token list and an active list record, and stores it; a bootstrap token named inside the minted `standards/` namespace is rejected before storage to keep its role unambiguous, while any other divergence from the recommended name only warns. `plan` and `apply` do not rotate GitHub App keys; create a replacement key on the App's settings page, atomically replace that owner's `private_key` in the protected machine-global broker store while preserving its `0600` mode, then update every intended SOPS destination, verify workflows can mint installation tokens, and revoke the old key manually. Root credentials live in the machine-global broker store (see Configuration); SOPS writes go through `sops edit` with a non-interactive editor, preserving comments and reusing the file's data key.
|
|
43
43
|
|
|
44
44
|
`structure --profile source` validates the standards source repository itself, which is deliberately not a consumer. The profile pins its intentional exceptions so they cannot drift silently: exact, ordered root gate scripts that run this CLI from the local checkout (`structure --profile source`, `dependabot --check` or `dependabot --write`, `github --check`, and the Turbo gate) instead of a recursive `standards check`, and a non-private published bin-only CLI workspace that carries a stable release SemVer and exactly maps the `standards` bin to `src/cli.ts` without `exports`. Every other consumer rule still applies.
|
|
45
45
|
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@davidvornholt/standards",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.24.1",
|
|
4
4
|
"description": "Bootstrap, synchronize, and validate davidvornholt/standards consumers",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -20,6 +20,7 @@
|
|
|
20
20
|
"LICENSE",
|
|
21
21
|
"README.md",
|
|
22
22
|
"src/cli.ts",
|
|
23
|
+
"src/contained-path.ts",
|
|
23
24
|
"src/creds-add.ts",
|
|
24
25
|
"src/creds-add-collision.ts",
|
|
25
26
|
"src/creds-add-cloudflare-commit.ts",
|
|
@@ -109,6 +110,7 @@
|
|
|
109
110
|
"src/github-settings-merge.ts",
|
|
110
111
|
"src/github-settings-parse.ts",
|
|
111
112
|
"src/github-settings.ts",
|
|
113
|
+
"src/json-file.ts",
|
|
112
114
|
"src/managed-files.ts",
|
|
113
115
|
"src/managed-path-ownership.ts",
|
|
114
116
|
"src/poller-approval.ts",
|
|
@@ -158,8 +160,13 @@
|
|
|
158
160
|
"src/sops-exec.ts",
|
|
159
161
|
"src/structure-check.ts",
|
|
160
162
|
"src/structure-profile.ts",
|
|
163
|
+
"src/structure-readme.ts",
|
|
164
|
+
"src/structure-secret-document.ts",
|
|
161
165
|
"src/structure-script.ts",
|
|
166
|
+
"src/structure-secrets.ts",
|
|
167
|
+
"src/structure-sops-envelope.ts",
|
|
162
168
|
"src/structure-workspace.ts",
|
|
169
|
+
"src/sync-policy.ts",
|
|
163
170
|
"src/yaml-emit.ts",
|
|
164
171
|
"src/yaml-parse.ts"
|
|
165
172
|
],
|
package/src/cli.ts
CHANGED
|
@@ -29,7 +29,7 @@ import { runDevEnv } from './dev-env';
|
|
|
29
29
|
import { CANONICAL_SETTINGS_FILE, LOCAL_SETTINGS_FILE } from './github-api';
|
|
30
30
|
import { runGithubApply, runGithubCheck } from './github-commands';
|
|
31
31
|
import { loadGithubSettings } from './github-settings';
|
|
32
|
-
import {
|
|
32
|
+
import { isRecord } from './github-settings-parse';
|
|
33
33
|
import {
|
|
34
34
|
findManagedFilesContainingBiomeDirectiveToken,
|
|
35
35
|
inspectManagedPath,
|
|
@@ -46,6 +46,11 @@ import { runPollerCommand } from './poller-commands';
|
|
|
46
46
|
import { collectStructureProblems } from './structure-check';
|
|
47
47
|
import type { StructureProfile } from './structure-profile';
|
|
48
48
|
import { hasSafeCommand } from './structure-script';
|
|
49
|
+
import {
|
|
50
|
+
inspectSyncPolicy,
|
|
51
|
+
readSyncPolicy,
|
|
52
|
+
selectedSyncRef,
|
|
53
|
+
} from './sync-policy';
|
|
49
54
|
|
|
50
55
|
const DEFAULT_UPSTREAM = 'github:davidvornholt/standards';
|
|
51
56
|
|
|
@@ -943,7 +948,7 @@ const runDoctor = async (consumer: string): Promise<boolean> => {
|
|
|
943
948
|
);
|
|
944
949
|
}
|
|
945
950
|
|
|
946
|
-
problems.push(...(await
|
|
951
|
+
problems.push(...(await inspectSyncPolicy(consumer)));
|
|
947
952
|
|
|
948
953
|
if (problems.length > 0) {
|
|
949
954
|
console.error(
|
|
@@ -1181,7 +1186,9 @@ const parseArgs = (argv: ReadonlyArray<string>): CliOptions => {
|
|
|
1181
1186
|
};
|
|
1182
1187
|
|
|
1183
1188
|
// Canonical monorepo structure gate: workspace and root script shapes,
|
|
1184
|
-
// internal versioning, `exports`, tsconfig inheritance,
|
|
1189
|
+
// internal versioning, `exports`, tsconfig inheritance, a11y wiring,
|
|
1190
|
+
// repo-owned workspace READMEs, and the undecrypted CI secrets contract
|
|
1191
|
+
// (SOPS shape mirroring plus the keys the synced workflows require).
|
|
1185
1192
|
// The `source` profile instead pins the standards template repository's own
|
|
1186
1193
|
// deliberate exceptions to the consumer contract.
|
|
1187
1194
|
const runStructure = async (
|
|
@@ -1229,82 +1236,6 @@ const runGithubCheckGate = (consumer: string): Promise<boolean> => {
|
|
|
1229
1236
|
return runGithubCheck(consumer);
|
|
1230
1237
|
};
|
|
1231
1238
|
|
|
1232
|
-
// Consumer-owned sync policy, checked in next to the canonical (read-only)
|
|
1233
|
-
// standards-sync workflow it configures — versioned and reviewable, unlike
|
|
1234
|
-
// repository Actions variables. All fields are optional; a missing file means
|
|
1235
|
-
// the defaults (track main, weekly auto-sync on).
|
|
1236
|
-
// autoSync false skips the scheduled workflow run; local CLI runs remain
|
|
1237
|
-
// the deliberate sync path.
|
|
1238
|
-
// ref tag, branch, or full commit sha to sync from instead of main.
|
|
1239
|
-
const POLICY_FILE = 'sync-standards.local.json';
|
|
1240
|
-
const LINE_BREAK = /[\r\n]/u;
|
|
1241
|
-
|
|
1242
|
-
type Policy = {
|
|
1243
|
-
readonly autoSync?: boolean;
|
|
1244
|
-
readonly ref?: string;
|
|
1245
|
-
};
|
|
1246
|
-
|
|
1247
|
-
const parsePolicy = (parsed: unknown): Policy => {
|
|
1248
|
-
if (!isRecord(parsed)) {
|
|
1249
|
-
throw new Error(`${POLICY_FILE} must be a JSON object`);
|
|
1250
|
-
}
|
|
1251
|
-
const unsupportedFields = Object.keys(parsed).filter(
|
|
1252
|
-
(field) => field !== 'autoSync' && field !== 'ref',
|
|
1253
|
-
);
|
|
1254
|
-
if (unsupportedFields.length > 0) {
|
|
1255
|
-
throw new Error(
|
|
1256
|
-
`${POLICY_FILE} contains unsupported field(s): ${unsupportedFields.join(', ')}`,
|
|
1257
|
-
);
|
|
1258
|
-
}
|
|
1259
|
-
if (parsed.autoSync !== undefined && typeof parsed.autoSync !== 'boolean') {
|
|
1260
|
-
throw new Error(`${POLICY_FILE} "autoSync" must be a boolean`);
|
|
1261
|
-
}
|
|
1262
|
-
if (
|
|
1263
|
-
parsed.ref !== undefined &&
|
|
1264
|
-
(!isNonEmptyString(parsed.ref) || LINE_BREAK.test(parsed.ref))
|
|
1265
|
-
) {
|
|
1266
|
-
throw new Error(
|
|
1267
|
-
`${POLICY_FILE} "ref" must be a non-empty single-line string`,
|
|
1268
|
-
);
|
|
1269
|
-
}
|
|
1270
|
-
return { autoSync: parsed.autoSync, ref: parsed.ref };
|
|
1271
|
-
};
|
|
1272
|
-
|
|
1273
|
-
const readPolicy = async (consumer: string): Promise<Policy> => {
|
|
1274
|
-
const raw = await readTextIfPresent(join(consumer, POLICY_FILE));
|
|
1275
|
-
if (raw === null) {
|
|
1276
|
-
return {};
|
|
1277
|
-
}
|
|
1278
|
-
let parsed: unknown;
|
|
1279
|
-
try {
|
|
1280
|
-
parsed = JSON.parse(raw);
|
|
1281
|
-
} catch (error) {
|
|
1282
|
-
throw new Error(`${POLICY_FILE} must contain valid JSON`, { cause: error });
|
|
1283
|
-
}
|
|
1284
|
-
return parsePolicy(parsed);
|
|
1285
|
-
};
|
|
1286
|
-
|
|
1287
|
-
const inspectPolicy = async (
|
|
1288
|
-
consumer: string,
|
|
1289
|
-
): Promise<ReadonlyArray<string>> => {
|
|
1290
|
-
try {
|
|
1291
|
-
await readPolicy(consumer);
|
|
1292
|
-
return [];
|
|
1293
|
-
} catch (error) {
|
|
1294
|
-
return [error instanceof Error ? error.message : String(error)];
|
|
1295
|
-
}
|
|
1296
|
-
};
|
|
1297
|
-
|
|
1298
|
-
// Policy validation is unconditional once the file exists. Selection happens
|
|
1299
|
-
// afterward: explicit refs win for remote sources, while local paths are used
|
|
1300
|
-
// as-is and ignore only the already-validated policy ref.
|
|
1301
|
-
const selectedRef = (
|
|
1302
|
-
src: string,
|
|
1303
|
-
explicitRef: string | undefined,
|
|
1304
|
-
policy: Policy,
|
|
1305
|
-
): string | undefined =>
|
|
1306
|
-
existsSync(src) ? explicitRef : (explicitRef ?? policy.ref);
|
|
1307
|
-
|
|
1308
1239
|
const runInitCommand = async (
|
|
1309
1240
|
consumer: string,
|
|
1310
1241
|
from: string | undefined,
|
|
@@ -1321,8 +1252,8 @@ const runInitCommand = async (
|
|
|
1321
1252
|
return;
|
|
1322
1253
|
}
|
|
1323
1254
|
const src = from ?? DEFAULT_UPSTREAM;
|
|
1324
|
-
const policy = await
|
|
1325
|
-
const source = resolveSource(src,
|
|
1255
|
+
const policy = await readSyncPolicy(consumer);
|
|
1256
|
+
const source = resolveSource(src, selectedSyncRef(src, ref, policy));
|
|
1326
1257
|
try {
|
|
1327
1258
|
const manifest = await loadManifest(
|
|
1328
1259
|
join(source.dir, 'sync-standards.json'),
|
|
@@ -1343,8 +1274,8 @@ const runSyncCommand = async (
|
|
|
1343
1274
|
join(consumer, 'sync-standards.json'),
|
|
1344
1275
|
);
|
|
1345
1276
|
const src = from ?? consumerManifest.upstream;
|
|
1346
|
-
const policy = await
|
|
1347
|
-
const source = resolveSource(src,
|
|
1277
|
+
const policy = await readSyncPolicy(consumer);
|
|
1278
|
+
const source = resolveSource(src, selectedSyncRef(src, ref, policy));
|
|
1348
1279
|
try {
|
|
1349
1280
|
const manifest = await loadManifest(
|
|
1350
1281
|
join(source.dir, 'sync-standards.json'),
|
|
@@ -0,0 +1,59 @@
|
|
|
1
|
+
import { lstatSync, realpathSync, type Stats } from 'node:fs';
|
|
2
|
+
import { isAbsolute, join, relative, sep } from 'node:path';
|
|
3
|
+
|
|
4
|
+
const isContained = (root: string, candidate: string): boolean => {
|
|
5
|
+
const fromRoot = relative(root, candidate);
|
|
6
|
+
return (
|
|
7
|
+
fromRoot !== '' &&
|
|
8
|
+
!isAbsolute(fromRoot) &&
|
|
9
|
+
fromRoot !== '..' &&
|
|
10
|
+
!fromRoot.startsWith(`..${sep}`)
|
|
11
|
+
);
|
|
12
|
+
};
|
|
13
|
+
|
|
14
|
+
const matchesKind = (stat: Stats, kind: 'directory' | 'file'): boolean =>
|
|
15
|
+
kind === 'file' ? stat.isFile() : stat.isDirectory();
|
|
16
|
+
|
|
17
|
+
// Validate the path component by component without following symlinks. Callers
|
|
18
|
+
// may read it only after this succeeds; the remaining check/read race is inside
|
|
19
|
+
// the package's non-hostile maintainer threat model.
|
|
20
|
+
export const isContainedPath = (
|
|
21
|
+
consumer: string,
|
|
22
|
+
rel: string,
|
|
23
|
+
kind: 'directory' | 'file',
|
|
24
|
+
): boolean => {
|
|
25
|
+
try {
|
|
26
|
+
if (
|
|
27
|
+
isAbsolute(rel) ||
|
|
28
|
+
rel.includes('\\') ||
|
|
29
|
+
lstatSync(consumer).isSymbolicLink()
|
|
30
|
+
) {
|
|
31
|
+
return false;
|
|
32
|
+
}
|
|
33
|
+
const parts = rel.split('/');
|
|
34
|
+
if (
|
|
35
|
+
parts.length === 0 ||
|
|
36
|
+
parts.some((part) => ['', '.', '..'].includes(part))
|
|
37
|
+
) {
|
|
38
|
+
return false;
|
|
39
|
+
}
|
|
40
|
+
const root = realpathSync(consumer);
|
|
41
|
+
let candidate = consumer;
|
|
42
|
+
for (const [index, part] of parts.entries()) {
|
|
43
|
+
candidate = join(candidate, part);
|
|
44
|
+
const stat = lstatSync(candidate);
|
|
45
|
+
const final = index === parts.length - 1;
|
|
46
|
+
const expectedKind = final ? kind : 'directory';
|
|
47
|
+
if (
|
|
48
|
+
stat.isSymbolicLink() ||
|
|
49
|
+
!isContained(root, realpathSync(candidate)) ||
|
|
50
|
+
!matchesKind(stat, expectedKind)
|
|
51
|
+
) {
|
|
52
|
+
return false;
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
return true;
|
|
56
|
+
} catch {
|
|
57
|
+
return false;
|
|
58
|
+
}
|
|
59
|
+
};
|
package/src/creds-dest.ts
CHANGED
|
@@ -4,7 +4,8 @@
|
|
|
4
4
|
|
|
5
5
|
import { lstatSync, readdirSync } from 'node:fs';
|
|
6
6
|
import { join } from 'node:path';
|
|
7
|
-
import {
|
|
7
|
+
import { isContainedPath } from './contained-path';
|
|
8
|
+
import { parseSopsKeyPath } from './creds-sops-structure';
|
|
8
9
|
import {
|
|
9
10
|
type BrokerStore,
|
|
10
11
|
type CloudflareBrokerAccount,
|
|
@@ -71,7 +72,7 @@ export const assertWritableSopsPath = (
|
|
|
71
72
|
const isYamlSecrets = (name: string): boolean =>
|
|
72
73
|
name.endsWith('.yaml') && !name.endsWith('.example.yaml');
|
|
73
74
|
const listDir = (consumer: string, rel: string): ReadonlyArray<string> => {
|
|
74
|
-
if (!
|
|
75
|
+
if (!isContainedPath(consumer, rel, 'directory')) {
|
|
75
76
|
return [];
|
|
76
77
|
}
|
|
77
78
|
try {
|
|
@@ -82,7 +83,7 @@ const listDir = (consumer: string, rel: string): ReadonlyArray<string> => {
|
|
|
82
83
|
};
|
|
83
84
|
const isHostTargetCandidate = (consumer: string, name: string): boolean => {
|
|
84
85
|
const rel = `infra/hosts/${name}`;
|
|
85
|
-
if (
|
|
86
|
+
if (isContainedPath(consumer, rel, 'directory')) {
|
|
86
87
|
return listDir(consumer, rel).includes('secrets.yaml');
|
|
87
88
|
}
|
|
88
89
|
try {
|
|
@@ -1,5 +1,3 @@
|
|
|
1
|
-
import { lstatSync, realpathSync, type Stats } from 'node:fs';
|
|
2
|
-
import { isAbsolute, join, relative, sep } from 'node:path';
|
|
3
1
|
import {
|
|
4
2
|
isAlias,
|
|
5
3
|
isMap,
|
|
@@ -42,57 +40,6 @@ export const parseSopsKeyPath = (
|
|
|
42
40
|
? segments
|
|
43
41
|
: null;
|
|
44
42
|
};
|
|
45
|
-
const isContained = (root: string, candidate: string): boolean => {
|
|
46
|
-
const fromRoot = relative(root, candidate);
|
|
47
|
-
return (
|
|
48
|
-
fromRoot !== '' &&
|
|
49
|
-
!isAbsolute(fromRoot) &&
|
|
50
|
-
fromRoot !== '..' &&
|
|
51
|
-
!fromRoot.startsWith(`..${sep}`)
|
|
52
|
-
);
|
|
53
|
-
};
|
|
54
|
-
const matchesKind = (stat: Stats, kind: 'directory' | 'file'): boolean =>
|
|
55
|
-
kind === 'file' ? stat.isFile() : stat.isDirectory();
|
|
56
|
-
export const isContainedSopsPath = (
|
|
57
|
-
consumer: string,
|
|
58
|
-
rel: string,
|
|
59
|
-
kind: 'directory' | 'file',
|
|
60
|
-
): boolean => {
|
|
61
|
-
try {
|
|
62
|
-
if (
|
|
63
|
-
isAbsolute(rel) ||
|
|
64
|
-
rel.includes('\\') ||
|
|
65
|
-
lstatSync(consumer).isSymbolicLink()
|
|
66
|
-
) {
|
|
67
|
-
return false;
|
|
68
|
-
}
|
|
69
|
-
const parts = rel.split('/');
|
|
70
|
-
if (
|
|
71
|
-
parts.length === 0 ||
|
|
72
|
-
parts.some((part) => ['', '.', '..'].includes(part))
|
|
73
|
-
) {
|
|
74
|
-
return false;
|
|
75
|
-
}
|
|
76
|
-
const root = realpathSync(consumer);
|
|
77
|
-
let candidate = consumer;
|
|
78
|
-
for (const [index, part] of parts.entries()) {
|
|
79
|
-
candidate = join(candidate, part);
|
|
80
|
-
const stat = lstatSync(candidate);
|
|
81
|
-
const final = index === parts.length - 1;
|
|
82
|
-
const expectedKind = final ? kind : 'directory';
|
|
83
|
-
if (
|
|
84
|
-
stat.isSymbolicLink() ||
|
|
85
|
-
!isContained(root, realpathSync(candidate)) ||
|
|
86
|
-
!matchesKind(stat, expectedKind)
|
|
87
|
-
) {
|
|
88
|
-
return false;
|
|
89
|
-
}
|
|
90
|
-
}
|
|
91
|
-
return true;
|
|
92
|
-
} catch {
|
|
93
|
-
return false;
|
|
94
|
-
}
|
|
95
|
-
};
|
|
96
43
|
const keyName = (node: unknown): string | null =>
|
|
97
44
|
isScalar(node) && typeof node.value === 'string' && node.tag === undefined
|
|
98
45
|
? node.value
|
package/src/creds-sops-value.ts
CHANGED
|
@@ -4,7 +4,8 @@
|
|
|
4
4
|
// unverifiable read, so the two negative answers must stay distinct: anything
|
|
5
5
|
// that is not a leaf we can read as a scalar string is unverifiable.
|
|
6
6
|
|
|
7
|
-
import {
|
|
7
|
+
import { isContainedPath } from './contained-path';
|
|
8
|
+
import { parseSopsKeyPath } from './creds-sops-structure';
|
|
8
9
|
import { runSops } from './sops-exec';
|
|
9
10
|
|
|
10
11
|
export type SopsStoredValueVerification =
|
|
@@ -51,7 +52,7 @@ export const verifySopsStoredValueWith = (
|
|
|
51
52
|
const { consumer, rel, dottedPath, expectedValue } = input;
|
|
52
53
|
const path = parseSopsKeyPath(dottedPath);
|
|
53
54
|
const problem = `could not verify stored SOPS value at ${dottedPath} in ${rel}`;
|
|
54
|
-
if (path === null || !
|
|
55
|
+
if (path === null || !isContainedPath(consumer, rel, 'file')) {
|
|
55
56
|
return { ok: false, problem };
|
|
56
57
|
}
|
|
57
58
|
const extract = path
|
package/src/creds-sops.ts
CHANGED
|
@@ -6,6 +6,7 @@ import { readFile } from 'node:fs/promises';
|
|
|
6
6
|
import { join } from 'node:path';
|
|
7
7
|
import process from 'node:process';
|
|
8
8
|
import { fileURLToPath } from 'node:url';
|
|
9
|
+
import { isContainedPath } from './contained-path';
|
|
9
10
|
import {
|
|
10
11
|
type SopsValueChange as EditorValueChange,
|
|
11
12
|
inspectSopsScalarStructure,
|
|
@@ -14,7 +15,6 @@ import {
|
|
|
14
15
|
} from './creds-sops-editor';
|
|
15
16
|
import {
|
|
16
17
|
inspectSopsStructure,
|
|
17
|
-
isContainedSopsPath,
|
|
18
18
|
type SopsShapeFailure,
|
|
19
19
|
} from './creds-sops-structure';
|
|
20
20
|
import { runSops } from './sops-exec';
|
|
@@ -45,7 +45,7 @@ export const readEncryptedKeys = async (
|
|
|
45
45
|
consumer: string,
|
|
46
46
|
rel: string,
|
|
47
47
|
): Promise<EncryptedKeysReadResult> => {
|
|
48
|
-
if (!
|
|
48
|
+
if (!isContainedPath(consumer, rel, 'file')) {
|
|
49
49
|
return {
|
|
50
50
|
ok: false,
|
|
51
51
|
kind: 'read-error',
|
|
@@ -68,7 +68,7 @@ export const inspectSopsScalarDestination = async (
|
|
|
68
68
|
rel: string,
|
|
69
69
|
dottedPath: string,
|
|
70
70
|
): Promise<SopsScalarDestinationResult> => {
|
|
71
|
-
if (!
|
|
71
|
+
if (!isContainedPath(consumer, rel, 'file')) {
|
|
72
72
|
return {
|
|
73
73
|
ok: false,
|
|
74
74
|
kind: 'read-error',
|
|
@@ -118,7 +118,7 @@ export const setSopsValues = (
|
|
|
118
118
|
rel: string,
|
|
119
119
|
changes: ReadonlyArray<EditorValueChange>,
|
|
120
120
|
): SopsWriteResult => {
|
|
121
|
-
if (!
|
|
121
|
+
if (!isContainedPath(consumer, rel, 'file')) {
|
|
122
122
|
return { ok: false, problem: `unsafe encrypted secrets target ${rel}` };
|
|
123
123
|
}
|
|
124
124
|
const result = runSops(['edit', rel], consumer, {
|
package/src/creds-target.ts
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
import {
|
|
1
|
+
import { isContainedPath } from './contained-path';
|
|
2
2
|
|
|
3
3
|
const SAFE_TARGET = /^[A-Za-z0-9](?:[A-Za-z0-9._-]*[A-Za-z0-9_-])?$/u;
|
|
4
4
|
|
|
@@ -26,8 +26,8 @@ export const resolveTargetRelResult = (
|
|
|
26
26
|
}
|
|
27
27
|
const host = `infra/hosts/${target}/secrets.yaml`;
|
|
28
28
|
const flat = `secrets/${target}.yaml`;
|
|
29
|
-
const hostExists =
|
|
30
|
-
const flatExists =
|
|
29
|
+
const hostExists = isContainedPath(consumer, host, 'file');
|
|
30
|
+
const flatExists = isContainedPath(consumer, flat, 'file');
|
|
31
31
|
if (hostExists && flatExists) {
|
|
32
32
|
return {
|
|
33
33
|
ok: false,
|
package/src/json-file.ts
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
import { readFile } from 'node:fs/promises';
|
|
2
|
+
import { isRecord } from './github-settings-parse';
|
|
3
|
+
|
|
4
|
+
export const readJsonFile = async (
|
|
5
|
+
path: string,
|
|
6
|
+
): Promise<Record<string, unknown> | null> => {
|
|
7
|
+
const raw = await readFile(path, 'utf8').catch(() => null);
|
|
8
|
+
if (raw === null) {
|
|
9
|
+
return null;
|
|
10
|
+
}
|
|
11
|
+
try {
|
|
12
|
+
const parsed = JSON.parse(raw) as unknown;
|
|
13
|
+
return isRecord(parsed) ? parsed : null;
|
|
14
|
+
} catch {
|
|
15
|
+
return null;
|
|
16
|
+
}
|
|
17
|
+
};
|
package/src/structure-check.ts
CHANGED
|
@@ -1,13 +1,16 @@
|
|
|
1
1
|
import { existsSync } from 'node:fs';
|
|
2
|
-
import { readdir
|
|
2
|
+
import { readdir } from 'node:fs/promises';
|
|
3
3
|
import { isAbsolute, join } from 'node:path';
|
|
4
4
|
import { isRecord } from './github-settings-parse';
|
|
5
|
+
import { readJsonFile } from './json-file';
|
|
5
6
|
import {
|
|
6
7
|
missingPublishedCliProblems,
|
|
7
8
|
rootScriptExpectations,
|
|
8
9
|
type StructureProfile,
|
|
9
10
|
} from './structure-profile';
|
|
11
|
+
import { collectWorkspaceReadmeProblems } from './structure-readme';
|
|
10
12
|
import { filteredTurboAliasProblem, hasSafeCommands } from './structure-script';
|
|
13
|
+
import { collectCiSecretsProblems } from './structure-secrets';
|
|
11
14
|
import { inspectWorkspace, type Workspace } from './structure-workspace';
|
|
12
15
|
|
|
13
16
|
const ROOT_FIXED_SCRIPTS = new Set(
|
|
@@ -17,20 +20,6 @@ const GLOB_SUFFIX = '/*';
|
|
|
17
20
|
const WORKSPACES_REQUIREMENT =
|
|
18
21
|
'package.json: "workspaces" must be a non-empty array of literal paths or one-level "<dir>/*" patterns';
|
|
19
22
|
|
|
20
|
-
const readJson = async (
|
|
21
|
-
path: string,
|
|
22
|
-
): Promise<Record<string, unknown> | null> => {
|
|
23
|
-
const raw = await readFile(path, 'utf8').catch(() => null);
|
|
24
|
-
if (raw === null) {
|
|
25
|
-
return null;
|
|
26
|
-
}
|
|
27
|
-
try {
|
|
28
|
-
const parsed = JSON.parse(raw) as unknown;
|
|
29
|
-
return isRecord(parsed) ? parsed : null;
|
|
30
|
-
} catch {
|
|
31
|
-
return null;
|
|
32
|
-
}
|
|
33
|
-
};
|
|
34
23
|
type ResolvedPattern = {
|
|
35
24
|
readonly dirs: ReadonlyArray<string>;
|
|
36
25
|
readonly problem: string | null;
|
|
@@ -116,7 +105,7 @@ const loadWorkspace = async (
|
|
|
116
105
|
if (!existsSync(path)) {
|
|
117
106
|
return { workspace: null, problem: null };
|
|
118
107
|
}
|
|
119
|
-
const manifest = await
|
|
108
|
+
const manifest = await readJsonFile(path);
|
|
120
109
|
if (manifest === null) {
|
|
121
110
|
return {
|
|
122
111
|
workspace: null,
|
|
@@ -158,9 +147,15 @@ export const collectStructureProblems = async (
|
|
|
158
147
|
consumer: string,
|
|
159
148
|
profile: StructureProfile,
|
|
160
149
|
): Promise<ReadonlyArray<string>> => {
|
|
161
|
-
const root = await
|
|
150
|
+
const [root, secretsProblems] = await Promise.all([
|
|
151
|
+
readJsonFile(join(consumer, 'package.json')),
|
|
152
|
+
collectCiSecretsProblems(consumer),
|
|
153
|
+
]);
|
|
162
154
|
if (root === null) {
|
|
163
|
-
return [
|
|
155
|
+
return [
|
|
156
|
+
'package.json must exist and contain a JSON object',
|
|
157
|
+
...secretsProblems,
|
|
158
|
+
];
|
|
164
159
|
}
|
|
165
160
|
const declaration = workspacePatternsOf(root);
|
|
166
161
|
const resolved = await Promise.all(
|
|
@@ -178,9 +173,16 @@ export const collectStructureProblems = async (
|
|
|
178
173
|
.map((ws) => ws.manifest.name)
|
|
179
174
|
.filter((name): name is string => typeof name === 'string'),
|
|
180
175
|
);
|
|
181
|
-
const inspections = await Promise.all(
|
|
182
|
-
|
|
183
|
-
|
|
176
|
+
const [inspections, readmeProblems] = await Promise.all([
|
|
177
|
+
Promise.all(
|
|
178
|
+
workspaces.map((ws) => inspectWorkspace(ws, workspaceNames, profile)),
|
|
179
|
+
),
|
|
180
|
+
collectWorkspaceReadmeProblems(
|
|
181
|
+
consumer,
|
|
182
|
+
profile,
|
|
183
|
+
workspaces.map((ws) => ws.rel),
|
|
184
|
+
),
|
|
185
|
+
]);
|
|
184
186
|
const requireA11y = inspections.some((i) => i.hasA11ySuite);
|
|
185
187
|
return [
|
|
186
188
|
...declaration.problems,
|
|
@@ -189,5 +191,7 @@ export const collectStructureProblems = async (
|
|
|
189
191
|
...missingPublishedCliProblems(profile, workspaces),
|
|
190
192
|
...inspectRootScripts(root, profile, requireA11y),
|
|
191
193
|
...inspections.flatMap((i) => [...i.problems]),
|
|
194
|
+
...readmeProblems,
|
|
195
|
+
...secretsProblems,
|
|
192
196
|
];
|
|
193
197
|
};
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
import { lstat, readFile } from 'node:fs/promises';
|
|
2
|
+
import { join } from 'node:path';
|
|
3
|
+
import { isContainedPath } from './contained-path';
|
|
4
|
+
import { readJsonFile } from './json-file';
|
|
5
|
+
import type { StructureProfile } from './structure-profile';
|
|
6
|
+
|
|
7
|
+
const SYNC_MANIFEST = 'sync-standards.json';
|
|
8
|
+
const TRAILING_SLASHES = /\/+$/u;
|
|
9
|
+
const MANIFEST_REQUIREMENT = `${SYNC_MANIFEST}: must contain a JSON object with a "paths" array of strings; the structure gate reads it to tell canonical workspaces from repo-owned ones`;
|
|
10
|
+
const MANIFEST_CONTAINMENT = `${SYNC_MANIFEST}: must be a contained regular file; symlinked paths are not allowed`;
|
|
11
|
+
|
|
12
|
+
type CanonicalRels = {
|
|
13
|
+
readonly rels: ReadonlySet<string>;
|
|
14
|
+
readonly problem: string | null;
|
|
15
|
+
};
|
|
16
|
+
|
|
17
|
+
// Canonical workspaces are synced from the standards template and cannot be
|
|
18
|
+
// fixed in a consumer checkout, so the consumer profile exempts them from
|
|
19
|
+
// repo-owned documentation rules. The source repository owns those workspaces,
|
|
20
|
+
// so the source profile exempts nothing.
|
|
21
|
+
const canonicalWorkspaceRels = async (
|
|
22
|
+
consumer: string,
|
|
23
|
+
profile: StructureProfile,
|
|
24
|
+
): Promise<CanonicalRels> => {
|
|
25
|
+
if (profile === 'source') {
|
|
26
|
+
return { rels: new Set(), problem: null };
|
|
27
|
+
}
|
|
28
|
+
const manifestPath = join(consumer, SYNC_MANIFEST);
|
|
29
|
+
const manifestInfo = await lstat(manifestPath).catch(() => null);
|
|
30
|
+
if (
|
|
31
|
+
manifestInfo !== null &&
|
|
32
|
+
!isContainedPath(consumer, SYNC_MANIFEST, 'file')
|
|
33
|
+
) {
|
|
34
|
+
return { rels: new Set(), problem: MANIFEST_CONTAINMENT };
|
|
35
|
+
}
|
|
36
|
+
const manifest = await readJsonFile(manifestPath);
|
|
37
|
+
const paths = manifest?.paths;
|
|
38
|
+
if (
|
|
39
|
+
!(
|
|
40
|
+
Array.isArray(paths) &&
|
|
41
|
+
paths.every((path): path is string => typeof path === 'string')
|
|
42
|
+
)
|
|
43
|
+
) {
|
|
44
|
+
return { rels: new Set(), problem: MANIFEST_REQUIREMENT };
|
|
45
|
+
}
|
|
46
|
+
return {
|
|
47
|
+
rels: new Set(paths.map((path) => path.replace(TRAILING_SLASHES, ''))),
|
|
48
|
+
problem: null,
|
|
49
|
+
};
|
|
50
|
+
};
|
|
51
|
+
|
|
52
|
+
const readmeProblem = async (
|
|
53
|
+
consumer: string,
|
|
54
|
+
rel: string,
|
|
55
|
+
): Promise<string | null> => {
|
|
56
|
+
const readmeRel = `${rel}/README.md`;
|
|
57
|
+
const readmePath = join(consumer, readmeRel);
|
|
58
|
+
const info = await lstat(readmePath).catch(() => null);
|
|
59
|
+
if (info !== null && !isContainedPath(consumer, readmeRel, 'file')) {
|
|
60
|
+
return `${rel}: README.md must be a contained regular file; symlinked paths are not allowed`;
|
|
61
|
+
}
|
|
62
|
+
const raw = await readFile(readmePath, 'utf8').catch(() => null);
|
|
63
|
+
return raw !== null && raw.trim() !== ''
|
|
64
|
+
? null
|
|
65
|
+
: `${rel}: repo-owned workspace must have a non-empty README.md`;
|
|
66
|
+
};
|
|
67
|
+
|
|
68
|
+
export const collectWorkspaceReadmeProblems = async (
|
|
69
|
+
consumer: string,
|
|
70
|
+
profile: StructureProfile,
|
|
71
|
+
rels: ReadonlyArray<string>,
|
|
72
|
+
): Promise<ReadonlyArray<string>> => {
|
|
73
|
+
const canonical = await canonicalWorkspaceRels(consumer, profile);
|
|
74
|
+
if (canonical.problem !== null) {
|
|
75
|
+
// Without the manifest the gate cannot tell canonical workspaces from
|
|
76
|
+
// repo-owned ones, so it fails on the manifest instead of guessing.
|
|
77
|
+
return [canonical.problem];
|
|
78
|
+
}
|
|
79
|
+
const problems = await Promise.all(
|
|
80
|
+
rels
|
|
81
|
+
.filter((rel) => !canonical.rels.has(rel))
|
|
82
|
+
.map((rel) => readmeProblem(consumer, rel)),
|
|
83
|
+
);
|
|
84
|
+
return problems.filter((problem): problem is string => problem !== null);
|
|
85
|
+
};
|
|
@@ -0,0 +1,170 @@
|
|
|
1
|
+
import { isRecord } from './github-settings-parse';
|
|
2
|
+
import {
|
|
3
|
+
hasCompleteSopsMetadata,
|
|
4
|
+
isEncryptedLeafValue,
|
|
5
|
+
isSopsEncryptedString,
|
|
6
|
+
looksEncrypted,
|
|
7
|
+
} from './structure-sops-envelope';
|
|
8
|
+
|
|
9
|
+
export type SecretLeaf = {
|
|
10
|
+
readonly id: string;
|
|
11
|
+
readonly path: string;
|
|
12
|
+
readonly segments: ReadonlyArray<string>;
|
|
13
|
+
readonly value: unknown;
|
|
14
|
+
};
|
|
15
|
+
|
|
16
|
+
export type SecretDocumentInspection = {
|
|
17
|
+
readonly leaves: ReadonlyMap<string, SecretLeaf>;
|
|
18
|
+
readonly problems: ReadonlyArray<string>;
|
|
19
|
+
};
|
|
20
|
+
|
|
21
|
+
export type RequiredSecretLeaf = {
|
|
22
|
+
readonly path: ReadonlyArray<string>;
|
|
23
|
+
readonly reason: string;
|
|
24
|
+
};
|
|
25
|
+
|
|
26
|
+
export const secretPathId = (segments: ReadonlyArray<string>): string =>
|
|
27
|
+
JSON.stringify(segments);
|
|
28
|
+
|
|
29
|
+
const collectLeaves = (
|
|
30
|
+
value: unknown,
|
|
31
|
+
segments: ReadonlyArray<string>,
|
|
32
|
+
into: Map<string, SecretLeaf>,
|
|
33
|
+
): void => {
|
|
34
|
+
if (
|
|
35
|
+
isRecord(value) &&
|
|
36
|
+
(segments.length === 0 || Object.keys(value).length > 0)
|
|
37
|
+
) {
|
|
38
|
+
for (const [key, child] of Object.entries(value)) {
|
|
39
|
+
collectLeaves(child, [...segments, key], into);
|
|
40
|
+
}
|
|
41
|
+
return;
|
|
42
|
+
}
|
|
43
|
+
const leaf = {
|
|
44
|
+
id: secretPathId(segments),
|
|
45
|
+
path: segments.join('.'),
|
|
46
|
+
segments,
|
|
47
|
+
value,
|
|
48
|
+
};
|
|
49
|
+
into.set(leaf.id, leaf);
|
|
50
|
+
};
|
|
51
|
+
|
|
52
|
+
const businessLeaves = (
|
|
53
|
+
mapping: Record<string, unknown>,
|
|
54
|
+
): ReadonlyMap<string, SecretLeaf> => {
|
|
55
|
+
const { sops: _sops, ...business } = mapping;
|
|
56
|
+
const leaves = new Map<string, SecretLeaf>();
|
|
57
|
+
collectLeaves(business, [], leaves);
|
|
58
|
+
return leaves;
|
|
59
|
+
};
|
|
60
|
+
|
|
61
|
+
const ambiguityProblems = (
|
|
62
|
+
rel: string,
|
|
63
|
+
leaves: ReadonlyMap<string, SecretLeaf>,
|
|
64
|
+
): ReadonlyArray<string> => {
|
|
65
|
+
const idsByDottedPath = new Map<string, Array<string>>();
|
|
66
|
+
for (const leaf of leaves.values()) {
|
|
67
|
+
const ids = idsByDottedPath.get(leaf.path) ?? [];
|
|
68
|
+
ids.push(leaf.id);
|
|
69
|
+
idsByDottedPath.set(leaf.path, ids);
|
|
70
|
+
}
|
|
71
|
+
return [...idsByDottedPath]
|
|
72
|
+
.filter(([, ids]) => ids.length > 1)
|
|
73
|
+
.map(
|
|
74
|
+
([path]) =>
|
|
75
|
+
`${rel}: key path "${path}" is ambiguous because direct dotted keys and nested mappings collide`,
|
|
76
|
+
);
|
|
77
|
+
};
|
|
78
|
+
|
|
79
|
+
export const inspectSecretDocument = (
|
|
80
|
+
rel: string,
|
|
81
|
+
mapping: Record<string, unknown>,
|
|
82
|
+
required: ReadonlyArray<RequiredSecretLeaf>,
|
|
83
|
+
): SecretDocumentInspection => {
|
|
84
|
+
const leaves = businessLeaves(mapping);
|
|
85
|
+
return {
|
|
86
|
+
leaves,
|
|
87
|
+
problems: [
|
|
88
|
+
...(hasCompleteSopsMetadata(mapping.sops)
|
|
89
|
+
? []
|
|
90
|
+
: [
|
|
91
|
+
`${rel}: incomplete top-level "sops" metadata; encrypt the file with SOPS before committing it`,
|
|
92
|
+
]),
|
|
93
|
+
...ambiguityProblems(rel, leaves),
|
|
94
|
+
...[...leaves.values()]
|
|
95
|
+
.filter((leaf) => !isEncryptedLeafValue(leaf.value))
|
|
96
|
+
.map(
|
|
97
|
+
(leaf) =>
|
|
98
|
+
`${rel}: value at "${leaf.path}" is not a complete SOPS-encrypted value; plaintext secret values must never be committed`,
|
|
99
|
+
),
|
|
100
|
+
...required.flatMap((requirement) => {
|
|
101
|
+
const leaf = leaves.get(secretPathId(requirement.path));
|
|
102
|
+
if (leaf === undefined) {
|
|
103
|
+
return [
|
|
104
|
+
`${rel}: missing required key "${requirement.path.join('.')}" — ${requirement.reason}`,
|
|
105
|
+
];
|
|
106
|
+
}
|
|
107
|
+
return isEncryptedLeafValue(leaf.value) &&
|
|
108
|
+
!isSopsEncryptedString(leaf.value)
|
|
109
|
+
? [
|
|
110
|
+
`${rel}: required key "${requirement.path.join('.')}" must be one SOPS-encrypted string scalar because its workflow resolves it as a string`,
|
|
111
|
+
]
|
|
112
|
+
: [];
|
|
113
|
+
}),
|
|
114
|
+
],
|
|
115
|
+
};
|
|
116
|
+
};
|
|
117
|
+
|
|
118
|
+
export const inspectExampleDocument = (
|
|
119
|
+
rel: string,
|
|
120
|
+
mapping: Record<string, unknown>,
|
|
121
|
+
): SecretDocumentInspection => {
|
|
122
|
+
const leaves = businessLeaves(mapping);
|
|
123
|
+
return {
|
|
124
|
+
leaves,
|
|
125
|
+
problems: [
|
|
126
|
+
...(mapping.sops === undefined
|
|
127
|
+
? []
|
|
128
|
+
: [`${rel}: must hold plaintext placeholders, not SOPS metadata`]),
|
|
129
|
+
...ambiguityProblems(rel, leaves),
|
|
130
|
+
...[...leaves.values()]
|
|
131
|
+
.filter((leaf) => looksEncrypted(leaf.value))
|
|
132
|
+
.map(
|
|
133
|
+
(leaf) =>
|
|
134
|
+
`${rel}: value at "${leaf.path}" looks SOPS-encrypted; replace it with a plaintext placeholder`,
|
|
135
|
+
),
|
|
136
|
+
],
|
|
137
|
+
};
|
|
138
|
+
};
|
|
139
|
+
|
|
140
|
+
type SecretShapeContext = {
|
|
141
|
+
readonly secretsRel: string;
|
|
142
|
+
readonly exampleRel: string;
|
|
143
|
+
readonly secrets: ReadonlyMap<string, SecretLeaf>;
|
|
144
|
+
readonly example: ReadonlyMap<string, SecretLeaf>;
|
|
145
|
+
readonly required: ReadonlyArray<RequiredSecretLeaf>;
|
|
146
|
+
};
|
|
147
|
+
|
|
148
|
+
export const secretShapeProblems = ({
|
|
149
|
+
secretsRel,
|
|
150
|
+
exampleRel,
|
|
151
|
+
secrets,
|
|
152
|
+
example,
|
|
153
|
+
required,
|
|
154
|
+
}: SecretShapeContext): ReadonlyArray<string> => {
|
|
155
|
+
const requiredIds = new Set(required.map(({ path }) => secretPathId(path)));
|
|
156
|
+
return [
|
|
157
|
+
...[...secrets.values()]
|
|
158
|
+
.filter((leaf) => !example.has(leaf.id))
|
|
159
|
+
.map(
|
|
160
|
+
(leaf) =>
|
|
161
|
+
`${exampleRel}: missing key "${leaf.path}" with the same mapping shape; mirror every ${secretsRel} key with a placeholder`,
|
|
162
|
+
),
|
|
163
|
+
...[...example.values()]
|
|
164
|
+
.filter((leaf) => !(secrets.has(leaf.id) || requiredIds.has(leaf.id)))
|
|
165
|
+
.map(
|
|
166
|
+
(leaf) =>
|
|
167
|
+
`${secretsRel}: missing key "${leaf.path}" with the same mapping shape; add the secret or delete the stale key from ${exampleRel}`,
|
|
168
|
+
),
|
|
169
|
+
];
|
|
170
|
+
};
|
|
@@ -0,0 +1,131 @@
|
|
|
1
|
+
import { lstat, readFile } from 'node:fs/promises';
|
|
2
|
+
import { join } from 'node:path';
|
|
3
|
+
import { isContainedPath } from './contained-path';
|
|
4
|
+
import { isRecord } from './github-settings-parse';
|
|
5
|
+
import {
|
|
6
|
+
inspectExampleDocument,
|
|
7
|
+
inspectSecretDocument,
|
|
8
|
+
type RequiredSecretLeaf,
|
|
9
|
+
secretShapeProblems,
|
|
10
|
+
} from './structure-secret-document';
|
|
11
|
+
import { readSyncPolicy } from './sync-policy';
|
|
12
|
+
import { parseYaml } from './yaml-parse';
|
|
13
|
+
|
|
14
|
+
const CI_SECRETS = 'secrets/ci.yaml';
|
|
15
|
+
const CI_EXAMPLE = 'secrets/ci.example.yaml';
|
|
16
|
+
const BROKER_REASON =
|
|
17
|
+
'the synced Standards sync workflow mints a branch token with Contents write and Workflows write plus a pull request token with Contents read and Pull requests write from ci.broker_app; provision it with "bun standards creds add github --dest ci:ci.broker_app"';
|
|
18
|
+
const NTFY_REQUIREMENT: RequiredSecretLeaf = {
|
|
19
|
+
path: ['ci', 'ntfy_topic_url'],
|
|
20
|
+
reason: 'the synced Notify pause workflow pushes to it',
|
|
21
|
+
};
|
|
22
|
+
const BROKER_REQUIREMENTS: ReadonlyArray<RequiredSecretLeaf> = [
|
|
23
|
+
{ path: ['ci', 'broker_app', 'app_id'], reason: BROKER_REASON },
|
|
24
|
+
{ path: ['ci', 'broker_app', 'private_key'], reason: BROKER_REASON },
|
|
25
|
+
];
|
|
26
|
+
|
|
27
|
+
type ParsedMapping = {
|
|
28
|
+
readonly mapping: Record<string, unknown> | null;
|
|
29
|
+
readonly problems: ReadonlyArray<string>;
|
|
30
|
+
};
|
|
31
|
+
|
|
32
|
+
const parseMapping = async (
|
|
33
|
+
consumer: string,
|
|
34
|
+
rel: string,
|
|
35
|
+
missingRequirement: string,
|
|
36
|
+
): Promise<ParsedMapping> => {
|
|
37
|
+
const path = join(consumer, rel);
|
|
38
|
+
const info = await lstat(path).catch(() => null);
|
|
39
|
+
if (info === null) {
|
|
40
|
+
return { mapping: null, problems: [missingRequirement] };
|
|
41
|
+
}
|
|
42
|
+
if (!isContainedPath(consumer, rel, 'file')) {
|
|
43
|
+
return {
|
|
44
|
+
mapping: null,
|
|
45
|
+
problems: [
|
|
46
|
+
`${rel}: must be a contained regular file; symlinked paths are not allowed`,
|
|
47
|
+
],
|
|
48
|
+
};
|
|
49
|
+
}
|
|
50
|
+
const raw = await readFile(path, 'utf8').catch(() => null);
|
|
51
|
+
if (raw === null) {
|
|
52
|
+
return { mapping: null, problems: [`${rel}: could not be read`] };
|
|
53
|
+
}
|
|
54
|
+
const parsed = parseYaml(raw, rel);
|
|
55
|
+
if (parsed.problem !== null) {
|
|
56
|
+
return { mapping: null, problems: [parsed.problem] };
|
|
57
|
+
}
|
|
58
|
+
if (!isRecord(parsed.value)) {
|
|
59
|
+
return { mapping: null, problems: [`${rel}: must be a YAML mapping`] };
|
|
60
|
+
}
|
|
61
|
+
return { mapping: parsed.value, problems: [] };
|
|
62
|
+
};
|
|
63
|
+
|
|
64
|
+
const requiredLeaves = async (
|
|
65
|
+
consumer: string,
|
|
66
|
+
): Promise<{
|
|
67
|
+
readonly required: ReadonlyArray<RequiredSecretLeaf>;
|
|
68
|
+
readonly problems: ReadonlyArray<string>;
|
|
69
|
+
}> => {
|
|
70
|
+
try {
|
|
71
|
+
const policy = await readSyncPolicy(consumer);
|
|
72
|
+
return {
|
|
73
|
+
required:
|
|
74
|
+
policy.autoSync === false
|
|
75
|
+
? [NTFY_REQUIREMENT]
|
|
76
|
+
: [NTFY_REQUIREMENT, ...BROKER_REQUIREMENTS],
|
|
77
|
+
problems: [],
|
|
78
|
+
};
|
|
79
|
+
} catch (error) {
|
|
80
|
+
return {
|
|
81
|
+
required: [NTFY_REQUIREMENT, ...BROKER_REQUIREMENTS],
|
|
82
|
+
problems: [error instanceof Error ? error.message : String(error)],
|
|
83
|
+
};
|
|
84
|
+
}
|
|
85
|
+
};
|
|
86
|
+
|
|
87
|
+
export const collectCiSecretsProblems = async (
|
|
88
|
+
consumer: string,
|
|
89
|
+
): Promise<ReadonlyArray<string>> => {
|
|
90
|
+
const [requirement, secretsFile, exampleFile] = await Promise.all([
|
|
91
|
+
requiredLeaves(consumer),
|
|
92
|
+
parseMapping(
|
|
93
|
+
consumer,
|
|
94
|
+
CI_SECRETS,
|
|
95
|
+
`${CI_SECRETS}: must exist as a SOPS-encrypted file; the synced CI workflows read ci.ntfy_topic_url and, when automatic sync is enabled, ci.broker_app from it`,
|
|
96
|
+
),
|
|
97
|
+
parseMapping(
|
|
98
|
+
consumer,
|
|
99
|
+
CI_EXAMPLE,
|
|
100
|
+
`${CI_EXAMPLE}: must exist and mirror the key shape of ${CI_SECRETS} with plaintext placeholders`,
|
|
101
|
+
),
|
|
102
|
+
]);
|
|
103
|
+
const secrets =
|
|
104
|
+
secretsFile.mapping === null
|
|
105
|
+
? null
|
|
106
|
+
: inspectSecretDocument(
|
|
107
|
+
CI_SECRETS,
|
|
108
|
+
secretsFile.mapping,
|
|
109
|
+
requirement.required,
|
|
110
|
+
);
|
|
111
|
+
const example =
|
|
112
|
+
exampleFile.mapping === null
|
|
113
|
+
? null
|
|
114
|
+
: inspectExampleDocument(CI_EXAMPLE, exampleFile.mapping);
|
|
115
|
+
return [
|
|
116
|
+
...requirement.problems,
|
|
117
|
+
...secretsFile.problems,
|
|
118
|
+
...exampleFile.problems,
|
|
119
|
+
...(secrets === null ? [] : secrets.problems),
|
|
120
|
+
...(example === null ? [] : example.problems),
|
|
121
|
+
...(secrets !== null && example !== null
|
|
122
|
+
? secretShapeProblems({
|
|
123
|
+
secretsRel: CI_SECRETS,
|
|
124
|
+
exampleRel: CI_EXAMPLE,
|
|
125
|
+
secrets: secrets.leaves,
|
|
126
|
+
example: example.leaves,
|
|
127
|
+
required: requirement.required,
|
|
128
|
+
})
|
|
129
|
+
: []),
|
|
130
|
+
];
|
|
131
|
+
};
|
|
@@ -0,0 +1,123 @@
|
|
|
1
|
+
import { isRecord } from './github-settings-parse';
|
|
2
|
+
|
|
3
|
+
const SOPS_ENVELOPE =
|
|
4
|
+
/^ENC\[AES256_GCM,data:(?<data>[A-Za-z0-9+/]+={0,2}),iv:(?<iv>[A-Za-z0-9+/]+={0,2}),tag:(?<tag>[A-Za-z0-9+/]+={0,2}),type:(?<type>bool|bytes|comment|float|int|str)\]$/u;
|
|
5
|
+
const SOPS_VERSION = /^\d+\.\d+\.\d+$/u;
|
|
6
|
+
const SOPS_SOURCE_CONTRACTS = [
|
|
7
|
+
['age', ['recipient', 'enc']],
|
|
8
|
+
['azure_kv', ['vault_url', 'name', 'version', 'created_at', 'enc']],
|
|
9
|
+
['gcp_kms', ['resource_id', 'created_at', 'enc']],
|
|
10
|
+
[
|
|
11
|
+
'hc_vault',
|
|
12
|
+
['vault_address', 'engine_path', 'key_name', 'created_at', 'enc'],
|
|
13
|
+
],
|
|
14
|
+
['kms', ['arn', 'created_at', 'enc']],
|
|
15
|
+
['pgp', ['created_at', 'enc', 'fp']],
|
|
16
|
+
] as const;
|
|
17
|
+
type SopsSource = (typeof SOPS_SOURCE_CONTRACTS)[number][0];
|
|
18
|
+
const SOPS_SOURCES = SOPS_SOURCE_CONTRACTS.map(([source]) => source);
|
|
19
|
+
const SOPS_SOURCE_FIELDS = new Map<SopsSource, ReadonlyArray<string>>(
|
|
20
|
+
SOPS_SOURCE_CONTRACTS,
|
|
21
|
+
);
|
|
22
|
+
|
|
23
|
+
const isCanonicalBase64 = (value: string): boolean => {
|
|
24
|
+
try {
|
|
25
|
+
return Buffer.from(value, 'base64').toString('base64') === value;
|
|
26
|
+
} catch {
|
|
27
|
+
return false;
|
|
28
|
+
}
|
|
29
|
+
};
|
|
30
|
+
|
|
31
|
+
const sopsEnvelopeType = (value: unknown): string | null => {
|
|
32
|
+
if (typeof value !== 'string') {
|
|
33
|
+
return null;
|
|
34
|
+
}
|
|
35
|
+
const match = SOPS_ENVELOPE.exec(value);
|
|
36
|
+
const groups = match?.groups;
|
|
37
|
+
return groups !== undefined &&
|
|
38
|
+
groups.data !== undefined &&
|
|
39
|
+
groups.iv !== undefined &&
|
|
40
|
+
groups.tag !== undefined &&
|
|
41
|
+
groups.type !== undefined &&
|
|
42
|
+
[groups.data, groups.iv, groups.tag].every(isCanonicalBase64)
|
|
43
|
+
? groups.type
|
|
44
|
+
: null;
|
|
45
|
+
};
|
|
46
|
+
|
|
47
|
+
export const isSopsEncryptedScalar = (value: unknown): boolean =>
|
|
48
|
+
sopsEnvelopeType(value) !== null;
|
|
49
|
+
|
|
50
|
+
export const isSopsEncryptedString = (value: unknown): boolean =>
|
|
51
|
+
sopsEnvelopeType(value) === 'str';
|
|
52
|
+
|
|
53
|
+
export const isEncryptedLeafValue = (value: unknown): boolean =>
|
|
54
|
+
isSopsEncryptedScalar(value) ||
|
|
55
|
+
(Array.isArray(value) &&
|
|
56
|
+
value.length > 0 &&
|
|
57
|
+
value.every(isSopsEncryptedScalar));
|
|
58
|
+
|
|
59
|
+
export const looksEncrypted = (value: unknown): boolean =>
|
|
60
|
+
(typeof value === 'string' && value.startsWith('ENC[')) ||
|
|
61
|
+
(Array.isArray(value) &&
|
|
62
|
+
value.some((item) => typeof item === 'string' && item.startsWith('ENC[')));
|
|
63
|
+
|
|
64
|
+
const hasRequiredStringFields = (
|
|
65
|
+
value: unknown,
|
|
66
|
+
fields: ReadonlyArray<string>,
|
|
67
|
+
): boolean =>
|
|
68
|
+
isRecord(value) &&
|
|
69
|
+
fields.every(
|
|
70
|
+
(field) => typeof value[field] === 'string' && value[field].length > 0,
|
|
71
|
+
);
|
|
72
|
+
|
|
73
|
+
const hasCompleteSource = (value: unknown, source: SopsSource): boolean => {
|
|
74
|
+
const fields = SOPS_SOURCE_FIELDS.get(source);
|
|
75
|
+
return (
|
|
76
|
+
fields !== undefined &&
|
|
77
|
+
Array.isArray(value) &&
|
|
78
|
+
value.length > 0 &&
|
|
79
|
+
value.every((entry) => hasRequiredStringFields(entry, fields))
|
|
80
|
+
);
|
|
81
|
+
};
|
|
82
|
+
|
|
83
|
+
const hasCompleteSources = (
|
|
84
|
+
value: Record<string, unknown>,
|
|
85
|
+
allowEmptyPlaceholders: boolean,
|
|
86
|
+
): boolean => {
|
|
87
|
+
const present = SOPS_SOURCES.filter((source) => value[source] !== undefined);
|
|
88
|
+
return (
|
|
89
|
+
present.length > 0 &&
|
|
90
|
+
present.some((source) => hasCompleteSource(value[source], source)) &&
|
|
91
|
+
present.every(
|
|
92
|
+
(source) =>
|
|
93
|
+
hasCompleteSource(value[source], source) ||
|
|
94
|
+
(allowEmptyPlaceholders &&
|
|
95
|
+
Array.isArray(value[source]) &&
|
|
96
|
+
value[source].length === 0),
|
|
97
|
+
)
|
|
98
|
+
);
|
|
99
|
+
};
|
|
100
|
+
|
|
101
|
+
const hasCompleteKeyGroups = (value: unknown): boolean =>
|
|
102
|
+
Array.isArray(value) &&
|
|
103
|
+
value.length > 0 &&
|
|
104
|
+
value.every((group) => isRecord(group) && hasCompleteSources(group, true));
|
|
105
|
+
|
|
106
|
+
const hasCompleteRecipients = (value: Record<string, unknown>): boolean => {
|
|
107
|
+
const hasDirectSource = SOPS_SOURCES.some(
|
|
108
|
+
(source) => value[source] !== undefined,
|
|
109
|
+
);
|
|
110
|
+
const hasKeyGroups = value.key_groups !== undefined;
|
|
111
|
+
return (
|
|
112
|
+
(hasDirectSource || hasKeyGroups) &&
|
|
113
|
+
(!hasDirectSource || hasCompleteSources(value, false)) &&
|
|
114
|
+
(!hasKeyGroups || hasCompleteKeyGroups(value.key_groups))
|
|
115
|
+
);
|
|
116
|
+
};
|
|
117
|
+
|
|
118
|
+
export const hasCompleteSopsMetadata = (value: unknown): boolean =>
|
|
119
|
+
isRecord(value) &&
|
|
120
|
+
typeof value.version === 'string' &&
|
|
121
|
+
SOPS_VERSION.test(value.version) &&
|
|
122
|
+
isSopsEncryptedScalar(value.mac) &&
|
|
123
|
+
hasCompleteRecipients(value);
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
import { existsSync } from 'node:fs';
|
|
2
|
+
import { lstat, readFile } from 'node:fs/promises';
|
|
3
|
+
import { join } from 'node:path';
|
|
4
|
+
import { isContainedPath } from './contained-path';
|
|
5
|
+
import { isNonEmptyString, isRecord } from './github-settings-parse';
|
|
6
|
+
|
|
7
|
+
export const SYNC_POLICY_FILE = 'sync-standards.local.json';
|
|
8
|
+
const LINE_BREAK = /[\r\n]/u;
|
|
9
|
+
|
|
10
|
+
export type SyncPolicy = {
|
|
11
|
+
readonly autoSync?: boolean;
|
|
12
|
+
readonly ref?: string;
|
|
13
|
+
};
|
|
14
|
+
|
|
15
|
+
const parseSyncPolicy = (parsed: unknown): SyncPolicy => {
|
|
16
|
+
if (!isRecord(parsed)) {
|
|
17
|
+
throw new Error(`${SYNC_POLICY_FILE} must be a JSON object`);
|
|
18
|
+
}
|
|
19
|
+
const unsupportedFields = Object.keys(parsed).filter(
|
|
20
|
+
(field) => field !== 'autoSync' && field !== 'ref',
|
|
21
|
+
);
|
|
22
|
+
if (unsupportedFields.length > 0) {
|
|
23
|
+
throw new Error(
|
|
24
|
+
`${SYNC_POLICY_FILE} contains unsupported field(s): ${unsupportedFields.join(', ')}`,
|
|
25
|
+
);
|
|
26
|
+
}
|
|
27
|
+
if (parsed.autoSync !== undefined && typeof parsed.autoSync !== 'boolean') {
|
|
28
|
+
throw new Error(`${SYNC_POLICY_FILE} "autoSync" must be a boolean`);
|
|
29
|
+
}
|
|
30
|
+
if (
|
|
31
|
+
parsed.ref !== undefined &&
|
|
32
|
+
(!isNonEmptyString(parsed.ref) || LINE_BREAK.test(parsed.ref))
|
|
33
|
+
) {
|
|
34
|
+
throw new Error(
|
|
35
|
+
`${SYNC_POLICY_FILE} "ref" must be a non-empty single-line string`,
|
|
36
|
+
);
|
|
37
|
+
}
|
|
38
|
+
return { autoSync: parsed.autoSync, ref: parsed.ref };
|
|
39
|
+
};
|
|
40
|
+
|
|
41
|
+
export const readSyncPolicy = async (consumer: string): Promise<SyncPolicy> => {
|
|
42
|
+
const path = join(consumer, SYNC_POLICY_FILE);
|
|
43
|
+
const info = await lstat(path).catch((error: unknown) => {
|
|
44
|
+
if (isRecord(error) && error.code === 'ENOENT') {
|
|
45
|
+
return null;
|
|
46
|
+
}
|
|
47
|
+
throw error;
|
|
48
|
+
});
|
|
49
|
+
if (info === null) {
|
|
50
|
+
return {};
|
|
51
|
+
}
|
|
52
|
+
if (!isContainedPath(consumer, SYNC_POLICY_FILE, 'file')) {
|
|
53
|
+
throw new Error(
|
|
54
|
+
`${SYNC_POLICY_FILE} must be a contained regular file; symlinked paths are not allowed`,
|
|
55
|
+
);
|
|
56
|
+
}
|
|
57
|
+
const raw = await readFile(path, 'utf8');
|
|
58
|
+
let parsed: unknown;
|
|
59
|
+
try {
|
|
60
|
+
parsed = JSON.parse(raw);
|
|
61
|
+
} catch (error) {
|
|
62
|
+
throw new Error(`${SYNC_POLICY_FILE} must contain valid JSON`, {
|
|
63
|
+
cause: error,
|
|
64
|
+
});
|
|
65
|
+
}
|
|
66
|
+
return parseSyncPolicy(parsed);
|
|
67
|
+
};
|
|
68
|
+
|
|
69
|
+
export const inspectSyncPolicy = async (
|
|
70
|
+
consumer: string,
|
|
71
|
+
): Promise<ReadonlyArray<string>> => {
|
|
72
|
+
try {
|
|
73
|
+
await readSyncPolicy(consumer);
|
|
74
|
+
return [];
|
|
75
|
+
} catch (error) {
|
|
76
|
+
return [error instanceof Error ? error.message : String(error)];
|
|
77
|
+
}
|
|
78
|
+
};
|
|
79
|
+
|
|
80
|
+
// Policy validation is unconditional once the file exists. Selection happens
|
|
81
|
+
// afterward: explicit refs win for remote sources, while local paths are used
|
|
82
|
+
// as-is and ignore only the already-validated policy ref.
|
|
83
|
+
export const selectedSyncRef = (
|
|
84
|
+
src: string,
|
|
85
|
+
explicitRef: string | undefined,
|
|
86
|
+
policy: SyncPolicy,
|
|
87
|
+
): string | undefined =>
|
|
88
|
+
existsSync(src) ? explicitRef : (explicitRef ?? policy.ref);
|