@toa.io/extensions.exposition 1.0.0-alpha.269 → 1.0.0-alpha.270

package/documentation/io.md

@@ -70,31 +70,32 @@ exposition:
         status: 404
       requests: 500
       interval: 30
-      cooldown: 30
 ```
 
-Requests are counted per `key`. Once `requests` are counted within `interval` seconds, further
-requests carrying that key are answered `429 Too Many Requests` until `cooldown` seconds pass.
-`interval` and `cooldown` are in seconds.
+Requests are metered per `key`. A key may spend `requests` at once, and earns them back at a rate of
+`requests` per `interval` seconds. A request carrying a key with nothing left to spend is answered
+`429 Too Many Requests`, with a `Retry-After` saying how long until one would be admitted again.
+`interval` is in seconds.
 
-`requests` is a budget for the **whole group** of gateway instances, not for one of them. Counting
-goes through Redis, so the instances converge on one number rather than each enforcing the limit
-separately. The number they act on is a lower bound: it never claims more requests than were really
-made, and it can lag by up to one interval. Precise per-request enforcement is not what this is for.
+So `requests` is the burst, and `requests / interval` the rate it is repaid at. There is no window to
+save a budget up in and spend twice across the edge of, and no lockout to sit out: a key that has
+overspent is admitted again as soon as it has earned a single request back, and a client that keeps
+asking meanwhile is not penalised for it.
 
-The lag is worth knowing about when `requests` is small. An instance sends what it has counted once
-an interval, and what the group made of it only comes back an interval later — so between its send
-and that reply, an instance is going on what it alone has counted since. A limit of `1` blocks on
-the first request whatever happens; a limit of a handful can be overshot in the first interval of a
-burst, before any instance has a group number to act on. Set `requests` for the rate you want to
-stop, not for an exact ceiling.
+`requests` is a budget for the **whole group** of gateway instances, not for one of them. Instances
+reconcile through Redis on a timer, so they converge on one number rather than each enforcing the
+limit separately. Between two of those, an instance goes on what it alone has spent — so the group
+can overshoot by what the other instances admit within that window, and by no more. The window is a
+tenth of `interval`, between a quarter of a second and two seconds, whatever `interval` is. Precise
+per-request enforcement is not what this is for.
 
-An instance that cannot reach Redis keeps throttling on what it has seen itself, and keeps serving.
+An instance that cannot reach Redis keeps throttling on what it has seen itself and keeps serving,
+and reports what it could not the next time it gets through.
 
 ### Key components
 
-What a request is counted against. Give one, or a list — a list keys on the combination, so
-`[route, ip]` counts each address separately on each route.
+What a request is metered against. Give one, or a list — a list keys on the combination, so
+`[route, ip]` meters each address separately on each route.
 
 - `ip` — the client address, read from `X-Forwarded-For` where it is set and not private,
   and from the connection otherwise.
@@ -107,11 +108,11 @@ What a request is counted against. Give one, or a list — a list keys on the co
 
 ### Conditions
 
-What is counted, as opposed to what it is counted against. A request that fails a condition is
-served and not counted. Conditions are evaluated against the response, so they cannot decide whether
-to block — only whether to count.
+What is metered, as opposed to what it is metered against. A request that fails a condition is served
+and costs nothing. Conditions are evaluated against the response, so they cannot decide whether to
+refuse a request — only whether it is charged for, once it has been answered.
 
-- `status: <code>` — count only responses with this status.
+- `status: <code>` — charge only for responses with this status.
 
 ```yaml
 io:throttle:
@@ -119,6 +120,8 @@ io:throttle:
   condition:
     status: 404
   requests: 20
-  interval: 60
-  cooldown: 600
+  interval: 600
 ```
+
+An address may probe for twenty missing paths at once, and thereafter for one every thirty seconds.
+Requests that find something are served and never counted against it.

package/documentation/notes/throttling.md

@@ -1,10 +1,11 @@
 # Decentralized Request Throttling
 
-> **Superseded.** The design below was never implemented. `io:throttle` now counts through
-> [comcount](https://github.com/temich/comcount), by way of the `stash` extension's `count`, which
-> solves the same problem the same way — per-process buffers, periodically reconciled through Redis,
-> never failing the request path — with the interval number derived from Redis' own clock instead of
-> each node's. This note is kept for the problem statement and the forces, which have not changed.
+> **Superseded.** The design below was never implemented. `io:throttle` is now a
+> [GCRA](https://en.wikipedia.org/wiki/Generic_cell_rate_algorithm) deciding in each process, with
+> its state periodically reconciled through Redis by way of the `stash` extension's `meter`. It
+> answers the same forces the same way — decide locally, reconcile in the background, never fail the
+> request path — and the section below records what it does instead. The problem statement and the
+> forces have not changed.
 
 ## Problem
 
@@ -86,3 +87,36 @@ When nodes are added or removed, the algorithm will adapt in the upcoming interv
    in a span on each node.
 2. Time desynchronization between nodes should be insignificant for the selected `INTERVAL` (i.e.,
    `INTERVAL` >> desync). See Extension point 3.
+
+## What was implemented instead
+
+Windows were dropped for a GCRA. A key holds one theoretical arrival time: the moment it would be
+back at zero if nothing more arrived. An admitted request pushes it `interval / requests`
+milliseconds further out, time drags it back, and a request is admitted while it stays within
+`interval` of now. So `requests` is what a key may spend at once and `requests / interval` the rate
+it earns back, with no window edge to burst across and no lockout to time out of — which is why
+`cooldown` is gone.
+
+Deciding reads nothing but a local map, so the request path stays free of I/O, exactly as force 1
+demands. What the other gateways have spent arrives on a timer, as **debt**: the milliseconds a key
+owes, draining at a millisecond a millisecond. Debt is the right thing to send for two reasons.
+
+1. It is a duration, not a moment, so processes exchanging it need not agree on the time — force 2,
+   answered by not needing an answer. The one clock that matters is Redis', read inside the script.
+2. It is additive in admissions, so a process reports only its own increments and Redis keeps the
+   running total. Nothing has to be divided by a number of nodes, which is as well: the gateway
+   cannot observe how many of it there are.
+
+The consequences worth knowing:
+
+- **Reconciling is decoupled from `interval`.** The timer runs at a tenth of `interval`, clamped
+  between 250ms and 2s, so a minute-long budget is no staler than a second-long one. The overshoot
+  is bounded by what the other instances admit within one tick — the caveat that replaces the old
+  note's `MAX_REQUESTS / N` per span.
+- **The cost is one round trip a tick**, not one per key. Every quota in the process flushes into a
+  single script call, which matters because a limiter keyed on `ip` watches as many keys as it has
+  clients.
+- **Redis keys expire on their own**, after the debt on them plus a grace, so nothing sweeps them.
+- **Failure degrades, as force 3 demands.** A tick that cannot reach Redis leaves its debt to the
+  next one rather than losing it; until then the instance throttles on what it alone has seen, and
+  keeps serving.

package/features/etag.feature

@@ -85,27 +85,77 @@ Feature: Optimistic concurrency control
           io:output: true
           GET: enumerate
       """
+    # sorted and bounded, or the collection does not enumerate the same way twice
     When the following request is received:
       """
-      GET /pots/ HTTP/1.1
+      GET /pots/?sort=id:asc&limit=2 HTTP/1.1
       host: nex.toa.io
       accept: application/yaml
       """
     Then the following reply is sent:
       """
       200 OK
-      etag: ${{ etag }}
+      etag: "${{ etag }}"
       """
+    # the same representation must be asked for, the tag identifies it
     When the following request is received:
       """
-      GET /pots/ HTTP/1.1
+      GET /pots/?sort=id:asc&limit=2 HTTP/1.1
       host: nex.toa.io
-      if-none-match: ${{ etag }}
+      accept: application/yaml
+      if-none-match: "${{ etag }}"
       """
     Then the following reply is sent:
       """
       304 Not Modified
-      etag: ${{ etag }}
+      etag: "${{ etag }}"
+      """
+
+  Scenario: `etag` of a reply that carries no version
+    Given the `echo` is running with the following manifest:
+      """yaml
+      exposition:
+        /:name:
+          io:output: true
+          GET: compute
+      """
+    When the following request is received:
+      """
+      GET /echo/Bob/ HTTP/1.1
+      host: nex.toa.io
+      accept: text/plain
+      """
+    Then the following reply is sent:
+      """
+      200 OK
+      etag: "2d6a12ffc0a952fbd09f8909de4e0e4b20ba2b906cd12ac22bbef4ee5bd9003e"
+
+      Hello Bob
+      """
+    When the following request is received:
+      """
+      GET /echo/Bob/ HTTP/1.1
+      host: nex.toa.io
+      accept: text/plain
+      if-none-match: "2d6a12ffc0a952fbd09f8909de4e0e4b20ba2b906cd12ac22bbef4ee5bd9003e"
+      """
+    Then the following reply is sent:
+      """
+      304 Not Modified
+      etag: "2d6a12ffc0a952fbd09f8909de4e0e4b20ba2b906cd12ac22bbef4ee5bd9003e"
+      """
+    # the tag identifies the representation, which is what `vary: accept` says
+    When the following request is received:
+      """
+      GET /echo/Bob/ HTTP/1.1
+      host: nex.toa.io
+      accept: application/yaml
+      if-none-match: "2d6a12ffc0a952fbd09f8909de4e0e4b20ba2b906cd12ac22bbef4ee5bd9003e"
+      """
+    Then the following reply is sent:
+      """
+      200 OK
+      etag: "360c52c694c418b4a793a4815e611206ab17fe04836351a65638a15d1491810b"
       """
 
   Scenario: Weak `etag`

package/features/io.throttle.feature

@@ -9,7 +9,6 @@ Feature: Request throttling
             key: [path]
             requests: 1
             interval: 1
-            cooldown: 1
           GET:
             endpoint: hello
       """
@@ -25,11 +24,13 @@ Feature: Request throttling
       """
       GET /echo/beacon/ HTTP/1.1
       """
+    # an emission is a second, which is what the budget takes to earn one back
     Then the following reply is sent:
       """
       429 Too Many Requests
+      retry-after: 1
       """
-    Then after 1 second
+    Then after 2 seconds
     When the following request is received:
       """
       GET /echo/beacon/ HTTP/1.1
@@ -49,7 +50,6 @@ Feature: Request throttling
               key: [route]
               requests: 1
               interval: 5
-              cooldown: 5
             GET:
               dev:stub:
                 hello: true
@@ -84,7 +84,6 @@ Feature: Request throttling
                 - segment: id
               requests: 1
               interval: 5
-              cooldown: 5
             GET:
               dev:stub:
                 hello: true
@@ -129,7 +128,6 @@ Feature: Request throttling
               key: [identity]
               requests: 1
               interval: 5
-              cooldown: 5
             endpoint: hello
       """
     And transient identity alice

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@toa.io/extensions.exposition",
-  "version": "1.0.0-alpha.269",
+  "version": "1.0.0-alpha.270",
   "description": "Toa Exposition",
   "author": "temich <tema.gurtovoy@gmail.com>",
   "homepage": "https://github.com/toa-io/toa#readme",
@@ -18,9 +18,9 @@
   },
   "dependencies": {
     "@simplewebauthn/server": "13.3.3",
-    "@toa.io/core": "1.0.0-alpha.266",
+    "@toa.io/core": "1.0.0-alpha.270",
     "@toa.io/generic": "1.0.0-alpha.254",
-    "@toa.io/schemas": "1.0.0-alpha.263",
+    "@toa.io/schemas": "1.0.0-alpha.270",
     "bcryptjs": "3.0.3",
     "content-type": "2.1.0",
     "error-value": "0.4.4",
@@ -31,7 +31,7 @@
     "minimatch": "10.2.6",
     "msgpackr": "2.1.0",
     "negotiator": "1.1.0",
-    "openspan": "1.0.0-alpha.264",
+    "openspan": "1.0.0-alpha.270",
     "paseto": "3.1.4"
   },
   "scripts": {
@@ -62,5 +62,5 @@
     },
     "testEnvironment": "node"
   },
-  "gitHead": "c2340ae684dcd997332075a7df5cbfe7872d1daa"
+  "gitHead": "94400bdd411b4c25074ffcf58184580ddd060268"
 }

package/schemas/io/throttle.cos.yaml

@@ -31,13 +31,12 @@ properties:
   requests:
     type: integer
     minimum: 1
-  interval: &interval
+  interval:
     description: Seconds
     type: integer
     minimum: 1
-  cooldown: *interval
+additionalProperties: false
 required:
   - key
   - requests
   - interval
-  - cooldown

package/source/Directive.test.ts

@@ -10,6 +10,7 @@ const families: Array<jest.MockedObjectDeep<DirectiveFamily>> = [
     name: 'foo',
     mandatory: true,
     create: jest.fn((_0: any, _1: any, _2: any) => generate() as any),
+    arrange: jest.fn(),
     preflight: jest.fn(),
     settle: jest.fn(),
     dispose: jest.fn()
@@ -18,6 +19,7 @@ const families: Array<jest.MockedObjectDeep<DirectiveFamily>> = [
     name: 'qux',
     mandatory: true,
     create: jest.fn((_0: any, _1: any, _2: any) => generate() as any),
+    arrange: jest.fn(),
     preflight: jest.fn(),
     settle: jest.fn(),
     dispose: jest.fn()
@@ -26,6 +28,7 @@ const families: Array<jest.MockedObjectDeep<DirectiveFamily>> = [
     name: 'bar',
     mandatory: false,
     create: jest.fn((_0: string, _1: any, _2: any) => generate() as any),
+    arrange: jest.fn(),
     preflight: jest.fn(),
     settle: jest.fn(),
     dispose: jest.fn()

package/source/Directive.ts

@@ -7,18 +7,27 @@ import type * as RTD from './RTD'
 export class Directives implements RTD.Directives {
   private readonly sets: RTD.DirectiveSet[]
 
+  /** the span of a stage depends only on the set, so it is built once per route */
+  private readonly spans: Spans[]
+
   public constructor (sets: RTD.DirectiveSet[]) {
     this.sets = sets
+    this.spans = sets.map((set) => ({
+      preflight: options(set, 'preflight'),
+      settle: options(set, 'settle')
+    }))
   }
 
   public async preflight (context: Context, parameters: RTD.Parameter[]): Promise<Output> {
     let output = null
 
-    for (const set of this.sets) {
+    for (let i = 0; i < this.sets.length; i++) {
+      const set = this.sets[i]
+
       if (set.family.preflight === undefined)
         continue
 
-      const out = await console.span(options(set, 'preflight'),
+      const out = await console.span(this.spans[i].preflight,
         async () => await set.family.preflight!(set.directives, context, parameters))
 
       if (out === null)
@@ -34,10 +43,13 @@ export class Directives implements RTD.Directives {
   }
 
   public async settle (context: Context, response: OutgoingMessage): Promise<void> {
-    for (const set of this.sets)
+    for (let i = 0; i < this.sets.length; i++) {
+      const set = this.sets[i]
+
       if (set.family.settle !== undefined)
-        await console.span(options(set, 'settle'),
+        await console.span(this.spans[i].settle,
           async () => await set.family.settle!(set.directives, context, response))
+    }
   }
 
   public dispose (): void {
@@ -106,6 +118,10 @@ export class DirectivesFactory implements RTD.DirectiveFactory {
     // The rest keep the order they were declared in, the sort being stable.
     sets.sort((a, b) => this.rank(a.family.name) - this.rank(b.family.name))
 
+    // whatever order a family needs among its own directives is fixed here, not per request
+    for (const set of sets)
+      set.family.arrange?.(set.directives)
+
     const directives = new Directives(sets)
 
     this.instances.push(directives)
@@ -135,6 +151,11 @@ function options (set: RTD.DirectiveSet, stage: 'preflight' | 'settle'): SpanOpt
   return options
 }
 
+interface Spans {
+  preflight: SpanOptions
+  settle: SpanOptions
+}
+
 export const shortcuts: RTD.syntax.Shortcuts = new Map([
   ['anonymous', 'auth:anonymous'],
   ['anyone', 'auth:anyone'],

package/source/Endpoint.ts

@@ -1,5 +1,4 @@
 import { Readable } from 'node:stream'
-import { createHash } from 'node:crypto'
 import { console } from 'openspan'
 import { Mapping } from './Mapping'
 import * as http from './HTTP'
@@ -125,16 +124,13 @@ export class Endpoint implements RTD.Endpoint {
     if (reply instanceof Readable)
       return false
 
-    const hash = `"${createHash('sha256').update(JSON.stringify(reply)).digest('hex')}"`
-
-    if (etag === hash) {
-      message.status = 304
-      message.headers.set('etag', etag)
-
-      return true
-    }
-
-    message.headers.set('etag', hash)
+    /*
+     * A reply that carries no version is tagged with a hash of its body. The body is
+     * serialized anyway when the response is written, so the tag is computed from what
+     * is actually sent rather than from a second serialization of the reply — which
+     * makes it specific to the negotiated representation, hence `vary: accept`.
+     */
+    message.etag = true
 
     return false
   }

package/source/Factory.ts

@@ -82,7 +82,22 @@ function configureLogs (): void {
 
   const tracesEnv = process.env[TRACES_ENV]
 
-  traces(tracesEnv === undefined ? {} : decode<TracesOptions>(tracesEnv))
+  traces(tracesEnv === undefined ? development() : decode<TracesOptions>(tracesEnv))
+}
+
+/**
+ * Tracing is off unless it is configured. The console exporter is a local development
+ * mechanism, so it is turned on for `toa dev` and for a boot trace the CLI has already
+ * asked for (`runtime/boot/src/span.js`), and nowhere else — a deployment that wants
+ * traces annotates `telemetry.traces.exporters`.
+ *
+ * The gateway boots without the telemetry extension, hence the copy of
+ * `extensions/telemetry/source/extension.ts`.
+ */
+function development (): TracesOptions {
+  const local = process.env.TOA_DEV === '1' || process.env.TOA_BOOT_TRACE === '1'
+
+  return local ? { exporters: { console: {} } } : {}
 }
 
 // eslint-disable-next-line @typescript-eslint/consistent-type-imports

package/source/HTTP/Context.ts

@@ -25,18 +25,23 @@ export class Context {
 
   private consumed = false
 
-  public constructor (authority: string, request: IncomingMessage, properties: Properties) {
+  // eslint-disable-next-line max-params
+  public constructor (authority: string, request: IncomingMessage, properties: Properties,
+    url: URL) {
     this.authority = authority
     this.request = request
 
     this.id = crypto.randomUUID()
-    this.url = new URL(request.url, `https://${request.headers.host}`)
+    // parsed by the server, which had to parse it anyway to know the request is valid
+    this.url = url
     this.timing = new Timing()
     this.debug = properties.debug
     this.log(request)
 
-    if (this.request.headers.accept !== undefined) {
-      const match = SUBTYPE.exec(this.request.headers.accept)
+    const accept = this.request.headers.accept
+
+    if (accept !== undefined) {
+      const match = SUBTYPE.exec(accept)
 
       if (match !== null) {
         const {
@@ -50,11 +55,10 @@ export class Context {
       }
     }
 
-    const negotiator = new Negotiator(this.request)
-    const mediaType = negotiator.mediaType(types)
+    const encoder = negotiate(this.request)
 
-    if (mediaType !== undefined)
-      this.encoder = formats[mediaType]
+    if (encoder !== undefined)
+      this.encoder = encoder
   }
 
   public async buffer (): Promise<Buffer> {
@@ -97,4 +101,31 @@ interface Properties {
   debug: boolean
 }
 
+/**
+ * Negotiation parses the header and sorts the candidates, and the value repeats:
+ * clients send one of a handful of `accept` strings. Bounded, the header is theirs.
+ */
+function negotiate (request: IncomingMessage): Format | undefined {
+  const accept = request.headers.accept ?? ''
+  const known = NEGOTIATED.get(accept)
+
+  if (known !== undefined)
+    return known === NONE ? undefined : known
+
+  const mediaType = new Negotiator(request).mediaType(types)
+  const encoder = mediaType === undefined ? undefined : formats[mediaType]
+
+  if (NEGOTIATED.size >= NEGOTIATED_LIMIT)
+    NEGOTIATED.clear()
+
+  NEGOTIATED.set(accept, encoder ?? NONE)
+
+  return encoder
+}
+
+/** distinguishes "negotiated to nothing" from "not negotiated yet" */
+const NONE = Symbol('not acceptable') as unknown as Format
+const NEGOTIATED = new Map<string, Format>()
+const NEGOTIATED_LIMIT = 1024
+
 const SUBTYPE = /^(?<type>\w{1,32})\/(vnd\.toa\.(?<subtype>\S{1,32})\+)(?<suffix>\S{1,32})$/

package/source/HTTP/Server.ts

@@ -92,10 +92,10 @@ export class Server extends Connector {
         response.destroy()
     })
 
-    const invalid = validate(request)
+    const url = parse(request)
 
-    if (invalid !== null) {
-      console.warn('Invalid request', errorAttributes(request, invalid))
+    if (url instanceof Error) {
+      console.warn('Invalid request', errorAttributes(request, url))
 
       response.writeHead(400).end()
 
@@ -129,8 +129,8 @@ export class Server extends Connector {
     const remote = trace(request.headers)
 
     const processing = remote === null
-      ? this.serve(request, response, authority)
-      : run(remote, async () => await this.serve(request, response, authority))
+      ? this.serve(request, response, authority, url)
+      : run(remote, async () => await this.serve(request, response, authority, url))
 
     processing.catch((error) => {
       console.error('Request processing failed', error)
@@ -140,9 +140,11 @@ export class Server extends Connector {
     })
   }
 
+  // eslint-disable-next-line max-params
   private async serve (request: http.IncomingMessage,
     response: http.ServerResponse,
-    authority: string): Promise<void> {
+    authority: string,
+    url: URL): Promise<void> {
     await console.span({
       name: `${request.method} ${request.url}`,
       kind: 'server',
@@ -151,7 +153,7 @@ export class Server extends Connector {
     }, async () => {
       response.setHeader('ray', current()!.traceId)
 
-      const context = new Context(authority, request as IncomingMessage, this.properties)
+      const context = new Context(authority, request as IncomingMessage, this.properties, url)
 
       await this.process!(context)
         .then(this.success(context, response))
@@ -201,6 +203,10 @@ export class Server extends Connector {
 
           const message: OutgoingMessage = { status: response.statusCode }
 
+          // eslint-disable-next-line max-depth
+          if (exception instanceof Exception && exception.headers !== undefined)
+            message.headers = exception.headers
+
           // eslint-disable-next-line max-depth
           if (context.encoder === null)
             message.body = undefined
@@ -226,11 +232,10 @@ export class Server extends Connector {
   }
 }
 
-function validate (request: http.IncomingMessage): null | Error {
+/** Parsing the URL is how a request is validated, so the `Context` is handed the result. */
+function parse (request: http.IncomingMessage): URL | Error {
   try {
-    void new URL(request.url!, `https://${request.headers.host}`)
-
-    return null
+    return new URL(request.url!, `https://${request.headers.host}`)
   } catch (error) {
     return error as Error
   }

package/source/HTTP/exceptions.ts

@@ -2,10 +2,14 @@ export class Exception extends Error {
   public readonly status: number
   public readonly body?: any
 
-  protected constructor (status: number, body?: any) {
+  /** Headers the status is not complete without, like `Retry-After` on a 429. */
+  public readonly headers?: Headers
+
+  protected constructor (status: number, body?: any, headers?: Headers) {
     super()
     this.status = status
     this.body = body
+    this.headers = headers
   }
 }
 
@@ -79,8 +83,10 @@ export class UnprocessableEntity extends ClientError {
 }
 
 export class TooManyRequests extends ClientError {
-  public constructor () {
-    super(429)
+  /** @param retry Seconds until the request would be admitted. */
+  public constructor (retry?: number) {
+    super(429, undefined,
+      retry === undefined ? undefined : new Headers({ 'retry-after': String(retry) }))
   }
 }