@jimhoyd/urlcode 0.3.0 → 0.4.0-alpha.2

package/docs/COMPLIANCE.md

@@ -49,11 +49,11 @@ A **project** rule runs once with:
 |---|---|
 | `document` | The parsed and validated `urlcode.yaml` (includes are merged into `routes`) |
 | `routes` | Route configuration by pattern, as written in YAML |
-| `plan` | `testPlan()`: `inventory[]` (`path`, `handler`, `methods`, `middleware`, `policies`, `state`), `policies` (the per-route describe map), `dynamicLinks` |
+| `plan` | `testPlan()`: `inventory[]` (`path`, `handler`, `methods`, `middleware`, `policies`, `state`), `policies` (the per-route describe map) |
 | `policies` | `effectivePolicies(document, route)` by pattern: the merged configuration of every policy on each route |
 | `origin` | The declared public origin, or `null` |
 | `target` | `node` unless the caller states another |
-| `host` | `{ requestLog, linkEvents, includeCode }` as declared for the deployment; `null` where undeclared |
+| `host` | `{ requestLog }` as declared for the deployment; `null` where undeclared |
 
 A **route** rule runs once per inventory entry and additionally receives
 `route` (the inventory entry), `config` (that route's YAML), `policy` (the
@@ -86,7 +86,6 @@ cache and compression secrets handling in `src/policies/cache.ts` and
 | `rfc6585/throttle-functions` | [RFC 6585](https://www.rfc-editor.org/rfc/rfc6585) §4 | medium | Every active function or middleware route has an effective `policies.throttle` | Declare a throttle on the route or the project |
 | `rfc9309/robots` | [RFC 9309](https://www.rfc-editor.org/rfc/rfc9309) | low | Some active route declares `policies.agents`, or an active `/robots.txt` `respond` route exists | Add a `/robots.txt` route or an agents deny list |
 | `rfc9110/expired-routes` | [RFC 9110](https://www.rfc-editor.org/rfc/rfc9110) §15.5.11 | info | Lists routes past `expires` that still answer 410 | Remove them once the 410 window has served its purpose |
-| `ops/management-private` | [Management security](MANAGEMENT-SECURITY.md) | info | Reminder when `dynamicLinks: true`: the management API and `/_urlcode` probes belong on a private bind | Run `links api` privately with an auth file; keep probes internal |
 
 ### `strict` (baseline plus)
 
@@ -102,15 +101,13 @@ cache and compression secrets handling in `src/policies/cache.ts` and
 
 These rules check deployment settings, so `audit` takes `--request-log` to
 declare the level the deployment uses (the audit process itself always logs
-nothing). The CLI cannot enable link events, so it declares `linkEvents:
-false`; embedders pass `host` themselves. The references are the runtime's
+nothing); embedders pass `host` themselves. The references are the runtime's
 own [logging guarantees](MONITORING.md): records carry no URL, query, header,
 body or binding, and `detailed` adds only the method and route pattern.
 
 | Rule | Standard | Severity | Checks | Remediation |
 |---|---|---|---|---|
 | `privacy/request-log-minimal` | [Monitoring](MONITORING.md), Log records | medium (`info` when undeclared) | `host.requestLog` is `minimal` | Use the default log unless per-route rates are required |
-| `privacy/link-events-off` | [Monitoring](MONITORING.md), The link event channel | medium; `high` with `includeCode` (`info` when undeclared and `dynamicLinks` is on) | The link event channel is off, or on without `includeCode` | Enable it only for a declared purpose; never disclose codes |
 | `privacy/detailed-log-parameters` | [Monitoring](MONITORING.md), Log records | low | With `detailed` logging, no active route takes parameters (records name the pattern and method, never values) | Keep `minimal` on parameterised deployments |
 
 ## Writing custom rules
@@ -159,9 +156,9 @@ Without any compliance flag the audit report is unchanged apart from
   "findings": [{ "rule": "rfc6585/throttle-functions", "severity": "medium", "route": "/hello/{name}",
                  "message": "…", "remediation": "…", "standard": { "name": "RFC 6585 …", "reference": "…", "section": "…" } }],
   "counts": { "high": 0, "medium": 4, "low": 11, "info": 1 }, "pass": true,
-  "evidence": { "routes": 21, "active": 19, "dynamicLinks": false, "policies": ["agents", "cache", "security", "throttle"],
+  "evidence": { "routes": 21, "active": 19, "policies": ["agents", "cache", "security", "throttle"],
                 "files": ["urlcode.yaml", "routes/code.yaml"], "origin": null, "target": "node",
-                "host": { "requestLog": "minimal", "linkEvents": false, "includeCode": null },
+                "host": { "requestLog": "minimal" },
                 "scope": "declared configuration and runtime facts; not a deployment or certification" } }
 ```
 
@@ -190,7 +187,7 @@ const report = await runCompliance(runtime, {
   rules: [], override: {}, disable: [],               // as a rules module would export them
   ignore: ['rfc9110/expired-routes'],
   origin: 'https://links.example',
-  host: { requestLog: 'minimal', linkEvents: false }, // what the deployment is configured with
+  host: { requestLog: 'minimal' }, // what the deployment is configured with
 });
 await runtime.close();
 ```

package/docs/CONDITIONS.md

@@ -0,0 +1,74 @@
+# Conditional routing
+
+Conditions are exact string comparisons over a bounded, portable input subset.
+A route's `match` is a conjunction: every declared query/header/cookie, host and
+method must match. A mismatch returns 404 without trying a less-specific path.
+Path precedence is unchanged; matching conditions is not authorization.
+
+```yaml
+version: "1"
+routes:
+  /beta:
+    match:
+      headers: {x-beta-user: "true"}
+    redirect: {url: https://beta.example.com}
+  /campaign:
+    conditional:
+      cases:
+        - match: {query: {source: newsletter}}
+          redirect: {url: https://example.com/newsletter}
+        - match: {query: {source: partner}}
+          redirect: {url: https://example.com/partner}
+      fallback:
+        respond: {text: "Choose a campaign"}
+```
+
+Use `conditional.cases` for multiple definitions at one path; duplicate YAML keys
+remain errors. Each case has a nonempty match and exactly one redirect/respond
+handler. The optional fallback has one of those handlers and no condition. No
+nested cases or case-local middleware/bindings/policies. Shared parameters,
+methods, response headers, middleware and policies belong to the route. A missing
+fallback returns 404. Up to 16 cases per route; pairs must be provably disjoint:
+at least one shared field must require different values. Ambiguous cases fail
+activation instead of relying on order. Cases are considered before fallback.
+
+Conditions admit query/header/cookie maps with 1–16 entries, names up to 128
+characters and values up to 1,024. Header names normalize to lowercase; duplicate
+case-insensitive names fail validation. Query strings are decoded once, have no
+type conversion or default substitution, and duplicate examined values return
+400. Examined duplicate header/cookie values return 400 when transport counts
+are available. Cookies compare unquoted wire values, without percent decoding;
+the Cookie header is bounded to 8 KiB. Missing values do not match empty strings.
+Standard authentication and transport headers cannot be predicates. There are no regex, numeric,
+geography, device, wildcard or arbitrary-code predicates.
+
+`match.host` compares the canonical authority of the **operator-configured public
+origin**, not client Host or forwarded headers. Configure the origin to use host
+conditions; one runtime does not infer multiple trusted public hosts. Methods are
+uppercase existing HTTP method tokens. A top-level match is checked before the
+route's method/handler execution; it may mask a method rejection with 404.
+
+Conditional results force `Cache-Control: no-store`; provider-specific CDN and
+surrogate cache directives cannot enable caching. Cache policies must be
+disabled or use no-store. This prevents one header/cookie branch from populating
+a shared origin or downstream cache. Route conditions and case coverage require
+explicit request fixtures; generated probes do not imply branch coverage.
+
+Self-hosted, AWS and Vercel execute the shared matcher. Node-adapter tests are
+local, not provider-deployment evidence; provider header coalescing limits still
+apply. Cloudflare refuses conditions until an artifact lowering and conformance
+tests exist. `urlcode capabilities` reports these distinctions. Operator plugins
+remain trusted host code; conditions add no guest authority.
+
+The [executable conditions project](../examples/conditions) includes explicit
+branch, fallback, duplicate and trusted-origin requests. Run:
+
+```sh
+urlcode test --project examples/conditions --origin https://conditions.example.test
+```
+
+Host names are canonical ASCII authorities (maximum 255 characters), with an
+optional nondefault port. IPv6 literals and IDN Unicode host spellings are not
+part of this initial condition syntax. Raw query comparisons remain separate
+from typed route inputs: a parameter default does not make an absent condition
+match, while all shared parameter validation still runs before a case handler.

package/docs/DEPLOYMENT-CHECKS.md

@@ -0,0 +1,108 @@
+# Verify a running deployment against the project
+
+`urlcode verify-deployment` answers one question: does the deployment behind
+`--target` behave the way this project declares? It starts the local snapshot
+(as `validate` does), sends a bounded set of HTTP requests to the target and
+compares what came back with what the snapshot says should come back. It reads
+responses only. It has no infrastructure access, uses no credential, follows no
+redirect and offers no `--insecure`.
+
+```bash
+urlcode verify-deployment --project ../my-links --target https://links.example
+urlcode verify-deployment --project ../my-links --target https://links.example \
+  --expect-routes 12 --compliance baseline --fail-on medium --timeout-ms 5000
+```
+
+Options: `--origin` (the public origin the local snapshot generates absolute
+URLs for; defaults to the target), `--expect-routes N`, `--expect-metrics`
+(the operator chose to expose `/_urlcode/metrics` publicly), `--timeout-ms`
+(per request, 100-120000, default 10000), `--fail-on high|medium|low|info|none`
+(default `high`), the `audit` compliance flags (`--compliance`,
+`--compliance-rules`, `--compliance-ignore`, `--compliance-warn`), and the
+usual `--policy` binding the local snapshot needs to start.
+
+## What it verifies
+
+Every check yields findings `{check, severity, route?, message, expected?,
+observed?}`. The report carries `checks` (assertions evaluated), `requests`
+(HTTP requests sent), `findings`, `counts` by severity, `notes` (what was
+deliberately not verified, such as compression delegated to a platform) and
+`pass`.
+
+| Check | What is compared | Severity |
+|---|---|---|
+| `probes` | `/_urlcode/health` and `/_urlcode/ready` answer 200 with `{status, version, routes}`; the deployed `version` equals the local snapshot's; the deployed route count equals the local count and `--expect-routes`; `/_urlcode/metrics` answers 404 unless `--expect-metrics` | high (metrics expected but absent: medium) |
+| `fixtures` | `tests/requests.json` and the generated native cases, sent exactly as `urlcode test --target` would: status, expected headers and expected body | high |
+| `security` | Per active literal route: every header of the effective security profile (`oshp`, `oshp-no-csp`, `set`, `unset`) with its exact value; a YAML `response.headers` value wins on a success, as it does in the runtime | high |
+| `cache` | `Cache-Control` (and `CDN-Cache-Control`) equal the strategy's emitted value; skipped where YAML or an asset handler owns the header and on function routes, whose handlers may answer `private` | medium |
+| `compression` | A second request with the declared `Accept-Encoding` on a response that is compressible (type, size at or above `minBytes`, no `no-transform`, no cookie, no secrets) carries `Content-Encoding`; on a delegated target it is noted, not checked | medium |
+| `agents` | A `User-Agent` from the denied bundled list answers the configured status on every route with an enforcing agents policy; project list files and bare patterns are not probed | high |
+| `throttle` | `RateLimit-Policy` carries the declared quota and window where the throttle is native | medium |
+| `site` | Generated `robots.txt`, `sitemap.xml`, `favicon`, `security.txt` and `llms.txt` answer 200 with the expected content type; `robots.txt` and `security.txt` bodies equal the generated file (a `Sitemap:` line missing means the deployment was started without `--origin`) | medium |
+| `methods` | A route that does not declare GET answers GET with 405 and `Allow`; a route with explicit `methods` answers OPTIONS with 405 and `Allow` | medium |
+| `head` | On a `respond` route, HEAD answers 200 with GET's `Content-Length` and no body | medium |
+| `errors` | An unmatched path (`/_urlcode-verify-<random>`) answers the runtime's 404 (`nosniff`, `no-store`) with the project-level security headers, which is also how a CDN error page or a different application shows itself | high |
+| `transport` | On an https target, `Strict-Transport-Security` is present wherever the profile emits it (the deployment must have been started with `--origin https://...`); a certificate Node rejects, a refused connection or a timeout on the health probe ends the run; a redirect whose destination is plain `http:` is reported | high (http: destination: low) |
+
+Bounds: fixtures plus generated cases, one to four requests per active literal
+route (GET, then HEAD, OPTIONS, an encoded GET or a denied User-Agent where the
+route calls for it), four probes. Concurrency is four for the route checks;
+fixtures run one at a time, as `test` does. A run refuses to start above 10000
+requests. Bodies are read up to 1 MiB and appear in the report only as the
+first 200 bytes of a failing assertion. Requests count against the
+deployment's throttle quotas: a quota smaller than the run turns later probes
+into refusals, which the report shows as fixture and header findings.
+
+## What it cannot verify
+
+- Anything not visible in a response: process settings, worker counts,
+  request logging, binding files, the proxy or CDN configuration, certificates
+  beyond Node's default chain and host-name check, DNS.
+- Routes with parameters, middleware or required inputs, except through the
+  fixtures the project supplies; the generated cases never invent business
+  data. Function bodies are verified only where a fixture asserts them.
+- Agents policies built from project list files or bare patterns, compression
+  delegated to a platform, throttle refusals (the run never exhausts a quota
+  on purpose) and cache hits (the origin cache is invisible from outside).
+- That the deployment stays this way: the report is a snapshot of one run.
+
+A passing report says the target answered like the local snapshot during the
+run. It is not a security assessment, a load test or a certification.
+
+## Exit codes
+
+`0` when no finding is at or above `--fail-on` (default `high`) and, when
+`--compliance` was given, the compliance report passes or `--compliance-warn`
+was set. `1` otherwise, and for an unreachable target, a rejected certificate,
+a target that is not a bare HTTP(S) origin, or invalid options. The report is
+printed as one JSON line after one `{"event":"finding",...}` line per finding.
+
+## How it complements audit, compliance and benchmark
+
+| Command | Runs against | Answers |
+|---|---|---|
+| `urlcode audit` | a local snapshot | are every route and method covered by passing checks, and do the counts reconcile |
+| `urlcode audit --compliance` | declared configuration | does the configuration meet the rule set |
+| `urlcode benchmark --target` | the deployment | how fast does it answer the workload |
+| `urlcode verify-deployment` | the deployment | does it answer the way the project declares |
+
+`audit` proves the project; `verify-deployment` proves the deployment is that
+project. Neither replaces the [operational drills](OPERATIONAL-PROOF.md).
+
+## A rollout gate
+
+```bash
+set -e
+urlcode audit --project . --expect-routes 12 --compliance baseline
+# deploy the candidate to the staging origin, then:
+urlcode verify-deployment --project . --target https://staging.links.example \
+  --expect-routes 12 --fail-on medium
+urlcode benchmark --project . --target https://staging.links.example --requests 500 --max-p95-ms 50
+# switch traffic, then verify the production origin the same way:
+urlcode verify-deployment --project . --target https://links.example --expect-routes 12
+```
+
+A version mismatch after the switch means traffic reaches a different
+snapshot: an old instance still serving, a cache in front of the origin, or a
+project revision that was never deployed. Roll back per
+[Operations](OPERATIONS.md#deployment-and-rollback-procedure).

package/docs/EGRESS.md

@@ -0,0 +1,125 @@
+# Bounded proxy and webhook transport
+
+Outbound requests use a host-owned HTTPS transport. Guests do not receive a fetch
+API. Project declarations express behavior; an external operator policy, pinned
+to the complete project revision, grants exact HTTPS origins separately to proxy
+and signal operations. A configured destination never grants its own authority.
+Binding injection uses existing explicitly granted secrets, resolved before the
+host transport receives a literal header value. Never log those values.
+
+The transport resolves the destination for each request, refuses any DNS response
+containing a nonpublic address, and pins its connection lookup to one validated
+answer. HTTPS certificate and hostname verification remain enabled. There is no
+connection pool, fallback address, redirect following, or automatic retry. Private,
+loopback, link-local, documentation, multicast, IPv4-mapped IPv6 and transition
+addresses are refused. IPv6 filtering is deliberately conservative: 2001::/16,
+2002::/16 and 3fff::/16 are refused along with addresses outside ordinary global
+unicast. Network-level egress controls remain useful defense in depth.
+
+Default limits per client are 16 simultaneous requests, five seconds including
+DNS, 1 MiB request and response bodies and 16 KiB headers. Excess concurrency is
+refused immediately. Requests use bounded buffering rather than streaming. Abort
+and shutdown destroy active HTTP requests; unresolved system DNS operations may
+finish in the background but cannot open a socket after cancellation. Those DNS
+operations retain a separate concurrency slot until they settle, so repeatedly
+timing out DNS cannot create an unbounded underlying lookup backlog. A shared
+64-operation DNS cap also spans all client instances and runtime reloads in the
+host Node isolate; closing a client does not release an unresolved DNS slot. Closing a
+client refuses future requests and drains its bounded outstanding promises.
+Errors carry only a fixed category, never a destination, request, header or secret.
+
+Proxy destinations have a literal HTTPS authority. Path placeholders are encoded
+as individual components; dot-segment values are refused. Query names and incoming
+and outgoing headers require explicit selection. Host, framing, hop-by-hop and
+proxy-authorization headers cannot be supplied. Ambient authorization, cookies,
+forwarding metadata (the entire `x-forwarded-*` family) and Set-Cookie cannot be selected. Explicit host-resolved
+Authorization injection is supported. Set-Cookie forwarding is deliberately
+unsupported because multiple cookie fields cannot safely be represented by the
+portable scalar header contract. Upstream redirects are returned as responses;
+Location is forwarded only if explicitly selected and is never fetched. Headers
+nominated by an incoming or upstream Connection field are removed even if selected.
+Header names are normalized to lowercase; array-valued upstream fields are omitted.
+An encoded request body requires explicitly selecting its Content-Encoding
+header; literal header injection cannot change or replace that coding. Bodies
+remain raw bytes, including content encoding; select Content-Encoding when
+forwarding compressed responses. No transparent decompression occurs.
+
+Webhook signals are best effort. The broker schedules work after the caller's
+synchronous emit operation, with eight concurrent deliveries by default and no
+queue. Saturation and closed brokers drop new events. Each event contains only
+version, declared route pattern, status and method: never the actual request URL,
+parameters, body, client identity, incoming headers or bindings. The webhook
+receives a POST with JSON. Only 2xx responses count as delivered; all other status
+codes and transport failures count as failed. Delivery order is unspecified,
+there are no retries or durability guarantees, and shutdown aborts outstanding
+work and awaits settlement. Counter snapshots report accepted, delivered, failed
+and dropped events without including destination data. Applications must not use
+these signals for guaranteed billing, audit retention or job execution.
+
+The transport is a new security-sensitive host bridge. Unit tests and CI are not
+an independent assessment or proof of hostile multi-tenant readiness. Public
+provider network deployments require separate operator-owned fixtures and evidence.
+
+## Project declarations and external grants
+
+```yaml
+version: "1"
+routes:
+  /items/{id}:
+    proxy:
+      url: https://api.example.com/items/{id}
+      query: [page]
+      requestHeaders: [accept]
+      responseHeaders: [content-type, content-encoding]
+      headers:
+        authorization: {secret: API_TOKEN}
+    parameters:
+      - {name: id, in: path, required: true, schema: {type: string}}
+    secrets:
+      API_TOKEN: {secret: UPSTREAM_AUTHORIZATION}
+    signals:
+      - url: https://hooks.example.com/events
+```
+
+An injected secret is the complete header value (for example an operator-provided
+Bearer value); interpolation and ambient credentials are not supported. Each
+proxy placeholder must name a declared string path parameter. Proxy routes refuse
+middleware in this initial implementation: middleware cannot safely authorize an
+already materialized upstream request. Use host request policies before egress.
+Proxy routes require cache disabled or `no-store`; explicit cache declarations
+cannot override this. Responses always remove CDN cache directives and receive
+`Cache-Control: no-store`. Compressed upstream bytes are refused if their
+Content-Encoding field was not explicitly selected; it cannot be overridden by
+route response headers.
+
+The external version-1 policy has the existing `projectSha256` and route grants:
+
+```json
+{"version":1,"projectSha256":"<reviewed revision hash>","routes":{"/items/{id}":{"secrets":["UPSTREAM_AUTHORIZATION"],"egress":{"proxy":["https://api.example.com"],"signals":["https://hooks.example.com"]}}}}
+```
+
+Use `permissions` to generate requested grants for review. Exact origin strings
+omit a trailing slash and the default 443 port. Nondefault ports are explicit.
+Proxy and signal purposes are independent even when they share an origin. Every
+route's grant is checked against the current revision before credentials, assets,
+workers or network clients are activated. Each runtime shares one bounded proxy
+client and one bounded signal client, with at most 64 origins per purpose.
+
+Signals emit when route execution and response policies produce a result,
+including guest middleware responses and returned error status codes. Thrown
+handler errors and host request-policy/plugin short circuits (including cache
+hits) do not emit. HEAD requests and generated readiness probes do not emit.
+The response never waits for webhook delivery. Counter events contain only
+accepted/delivered/failed/dropped outcomes and counts and are exposed through
+runtime metrics and the Prometheus `signals_total` series. A closed runtime stops
+new requests, aborts egress and drains bounded pending work. Client disconnects
+are not currently propagated into the runtime transport; the five-second deadline
+still applies. All non-self-hosted targets currently refuse proxy and signals.
+See the [executable example](../examples/egress/README.md).
+
+Revision hashes also include declared project policies, profiles and site
+configuration. Changing an inherited pre-egress restriction invalidates grants
+just like changing a route or function source. Projects with these declarations
+must regenerate and review their operator policy after upgrading to this hash
+coverage; an old grant is intentionally refused rather than silently retaining
+network authority under changed behavior.