caliper-timings 0.0.0-stage → 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,26 @@
+# Changelog
+
+## 0.2.0 — 2026-11-12
+
+### Added
+
+- **`run.prompt(messages)`: prompt sizes.** Records each message's role and the size of its text in bytes (UTF-8), in order, in the record's new optional `prompt` field (`{ "messages": [{ "role": "system", "bytes": 2310 }, …] }`). Never the text: a role that isn't a lowercase name of up to 32 letters or `_` is recorded as `"?"`, content that isn't a string gets `bytes: null` (never estimated), and problems name the message by number only. At most 1000 messages. Never throws.
+- **SPEC v1: optional `prompt` field** ([Prompt sizes](SPEC.md#prompt-sizes)). Not breaking: a record without it is unchanged. caliper-ai 0.4.0's record reader (from the release commit) was run once on a record with `prompt`: it read the marks and ignored the field.
+- `isValidRole` and the `PromptMessageSize` type are exported.
+- **Known limit:** a role of up to 32 lowercase letters or `_` is recorded as it is, so text an app puts in a role in that form is recorded too. Message content never is.
+- **Tests that `run.prompt()` never carries text or changes the app** (a marker in every part of a prompt; a Proxy that records writes; lists that throw when read; byte-identical responses over HTTP; no reads when unmarked), each checked by breaking the code.
+- **Measured cost of `run.prompt()`:** median 2.5 µs per call for a Quill-sized prompt (8184 bytes), 10.0 µs for 40 messages, on Node 22.22 (Apple M2).
+
+## 0.1.0 — 2026-11-04
+
+### Added
+
+- **Core:** `CaliperRun`, `MemoryStore`, `tokenMatches`, `isValidRunId`, `isValidMarkName` and the `TimingRecord` type, ported from Quill's backend. They now use only erasable TypeScript, so Node 22 runs the source directly.
+  - **Mark names** are now app-chosen, following SPEC v1: `<name>Start` / `<name>End` pairs, plus the reserved `modelCallSent` and `firstChunk`. An invalid name is skipped and reported in the record's new `problems` field.
+  - **Nothing throws into the app's request** (SPEC: never change the app). A store that throws synchronously no longer reaches the request; Quill's version only caught rejections. A store failure on the timings endpoint answers 500.
+  - **`MemoryStore`** now also deletes an expired record when a `get` finds it. Before, only the next `put` did.
+  - **`watchFirstChunk`** skips chunks with no bytes.
+- **`caliper-timings/node`:** `createTimings({ token, store })`, with `handle(req, res)` for the timings endpoint and `start(req, res)` for a route.
+- **`SPEC.md`:** the protocol, v1, with the rule that an implementation never changes the app's response or errors, and the `problems` field.
+- **First published release.** The package ships the compiled JavaScript and type declarations (`dist/`), with this changelog, the README, SPEC.md and the license.
+- **Measured cost** in SPEC.md and the README: on `examples/node-memory`, marked requests median 1.2 ms slower before the first byte (20 pairs), unmarked requests median 0.0 ms against the pre-wiring server (20 pairs). Cause not measured.

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Jiya Agrawal
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -1,3 +1,137 @@
-# Temporary Holding Version
+# caliper-timings
 
-This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
+The server side of [Caliper](https://github.com/Jia-A/caliper)'s server timings. Caliper (`caliper-ai`) measures an AI feature in a real browser. With this on your server, it can also show what happened inside one marked request: how long it waited on the database, on the model, or on any step you mark.
+
+Light enough to stay in production, deployed dark:
+
+- No runtime dependencies.
+- The core uses only `Date.now()`, Web Streams and `crypto.subtle`.
+- Off unless `CALIPER_TOKEN` is set.
+- Unmarked requests are never touched.
+
+The wire format is in [SPEC.md](SPEC.md), so other languages can implement it too.
+
+## Install
+
+Two steps, in two places. Step 2 is optional.
+
+### 1. On your machine: `caliper-ai`
+
+The machine you run Caliper from (your laptop, or CI). It's the CLI that marks a request and reads the timings back:
+
+```sh
+npm install -g caliper-ai
+npx playwright install chromium
+```
+
+See the [caliper-ai README](https://github.com/Jia-A/caliper/tree/main/packages/cli#readme).
+
+### 2. In your server's project: `caliper-timings` (optional)
+
+Only if you want server timings. Install it in the project of the app Caliper measures, then wire it in as below:
+
+```sh
+npm install caliper-timings
+```
+
+Without it (or without `CALIPER_TOKEN` set on the server), Caliper still times everything it sees from the browser. Requires Node.js 20 or newer.
+
+## Node (`node:http`)
+
+```js
+import { createServer } from "node:http";
+import { createTimings, MemoryStore } from "caliper-timings/node";
+
+const timings = createTimings({ token: process.env.CALIPER_TOKEN, store: new MemoryStore() });
+
+createServer(async (req, res) => {
+  if (await timings.handle(req, res)) return; // GET /__caliper/timings/:id; false for everything else
+
+  if (req.method === "POST" && req.url === "/api/generate") {
+    const run = await timings.start(req, res); // null unless this request is marked
+    run?.mark("dbStart");
+    const row = await db.query(/* … */);
+    run?.mark("dbEnd");
+    const messages = [{ role: "system", content: SYSTEM }, { role: "user", content: row.text }];
+    run?.prompt(messages); // each message's role and size in bytes, never its text
+    run?.mark("modelCallSent");
+    res.writeHead(200, { "content-type": "text/event-stream" });
+    for await (const chunk of model.stream(/* … */)) res.write(chunk); // the first write with bytes: firstChunk
+    res.end();
+    return;
+  }
+  // … your other routes and your 404
+}).listen(8787);
+```
+
+- **`handle(req, res)`** answers `GET /__caliper/timings/:id` in SPEC.md's check order and returns `true`. For any other request it returns `false` and doesn't touch the response. With no token set it returns `false` for every request, so your own 404 answers.
+- **`start(req, res)`** returns a `CaliperRun` only when the request carries a valid `x-caliper-run` and the right `x-caliper-token`. Otherwise it returns `null` and the request and response are left exactly as they were. Call it first thing in the route: marks are measured from the moment `start()` was called.
+- **`run.mark(name)`** records ms since then. Name spans with `<name>Start` / `<name>End` (`dbStart` / `dbEnd`); Caliper shows each pair as a span called `<name>`. `modelCallSent` and `firstChunk` are reserved. A name outside `^[A-Za-z][A-Za-z0-9_]{0,63}$` is skipped, not thrown, and Caliper prints it under "problems reported by the server".
+- **`run.prompt(messages)`** records each message's role and its size in bytes (UTF-8), in order, for Caliper's prompt sizes. Pass the same list you send to the model, as `{ role, content }` objects, just before the model call. With a separate system text (Anthropic's API), pass it as the first message. It never records the text. A role that isn't a lowercase name (`system`, `user`, `assistant`, …) is recorded as `"?"`, and content that isn't a string gets no size (`null`), with a problem saying which message, by number. It reads the list and never changes it. A later call replaces an earlier one. See [SPEC.md](SPEC.md#prompt-sizes).
+- **When the record is saved:** at the first `res.write` with bytes, with `firstChunk` marked then. Otherwise when the response closes, whether it finished or the client left. `status` is `res.statusCode`, or `null` if no headers were sent. Every write goes through unchanged.
+
+### It never changes your app
+
+A marked request succeeds or fails exactly as it would unmarked; that's the protocol's [rule](SPEC.md#the-rule-never-change-the-app). Nothing in the wrapper throws into your request:
+- **`start()`** returns `null` if anything inside it fails, and the request runs unmarked.
+- **`mark()` and `save()`** never throw or reject.
+- **A store that throws or rejects** on save is logged to `console.error`, and the record is dropped.
+- **A URL `handle()` can't parse** is left to your app.
+- **A store failure while Caliper reads the timings endpoint** answers that request with 500.
+
+### MemoryStore
+
+`MemoryStore` keeps records in a `Map`, for 10 minutes after each write.
+
+**Use it for single-process servers only.** On Workers, or several processes behind a load balancer, each one has its own `Map`, so Caliper's read can land where the write never happened (`404`).
+
+**Cleanup:** every `put` deletes every expired record first, and a `get` that finds an expired record deletes it. After a write, the store holds only records written in the last 10 minutes. If writes stop, what's left stays until the next `put` or `get`. That's bounded by the last 10 minutes of marked requests, not a growing leak. Only requests carrying the right token create records.
+
+## Core (`caliper-timings`)
+
+The core works in any runtime with Web Streams and `crypto.subtle`. It exports:
+
+- `CaliperRun`, with `mark`, `prompt`, `save`, and `watchFirstChunk(stream)` for frameworks that return a `ReadableStream`;
+- `MemoryStore` and the `TimingStore` interface;
+- `tokenMatches`, `isValidRunId`, `isValidMarkName`, `isValidRole`;
+- the header and path constants;
+- the `TimingRecord` and `PromptMessageSize` types.
+
+There's no framework adapter other than Node yet.
+
+## Measured cost
+
+The rule covers status, headers, bytes and errors, not time. This is what the reference implementation (`caliper-timings/node`) measured in time, on [`examples/node-memory`](https://github.com/Jia-A/caliper/tree/main/examples/node-memory): a single-process `node:http` app on 127.0.0.1, Node 22.22, one macOS machine. The client is `examples/node-memory/measure.mjs`, timing on its own clock from sending the request to the first body chunk arriving, one request at a time, each on a new connection.
+
+- **Marked requests: about 1.2 ms slower before the first byte.** 20 marked and 20 unmarked requests to the same server, alternating, compared in pairs: the marked one was later by median 1.2 ms (min 0.1, max 4.1). The difference is already there when the headers arrive (paired median 1.4 ms); between headers and first chunk it's 0.1 ms. What causes it hasn't been measured.
+- **`run.prompt()`: microseconds per call, measured on its own.** On Node 22.22 (Apple M2), 50 batches of 1000 calls each: a Quill-sized prompt (2 messages, 8184 bytes) took median 2.5 µs per call (min 2.4, max 4.6), and a 40-message chat (500 bytes each) median 10.0 µs (min 9.7, max 10.5). That's the call alone, not a request's time end to end, and it was measured on one machine, not on Workers. An unmarked request doesn't call it: with `run?.prompt(messages)`, `run` is `null`, so the list isn't even built (tested).
+- **Unmarked requests: no difference measured.** 20 unmarked requests to the wired server (`CALIPER_TOKEN` set) alternated with 20 to the same app before it was wired: paired difference median 0.0 ms (min −2.3, max 5.4).
+
+That is one app, one machine and 20 pairs each. Other runtimes, frameworks, stores and loads haven't been measured.
+
+## Tested
+
+`npm test` runs Node 22.18+ directly on the TypeScript source. It covers:
+
+- **The endpoint's check order**, over a real socket on 127.0.0.1: 404 with no or empty token (the app's own), 401 for a missing, wrong or non-`Bearer` token before the store is read, 400 for malformed ids including Caliper's `caliper.precheck`, 404 for no record, 200 with the record, and `no-store` on every answer.
+- **Which requests qualify:** marked requests get the same status, headers and bytes as unmarked ones. Requests missing a header, with a wrong token or a malformed id, or on a server with no token, record nothing.
+- **When the record is saved:** at the first write with bytes (an empty write doesn't count), at close for `res.end(body)` replies, and at close with `status: null` when the client leaves before headers.
+- **Never changing the app:** with a store whose `put` throws, or one whose `put` rejects, the marked response is byte-identical to the unmarked one and nothing escapes as an unhandled error. With a broken `get`, the endpoint answers 500 and app requests are untouched. A URL that doesn't parse is left to the app. A `save()` with a throwing store or `defer` neither throws nor rejects. Each of these was checked by removing the guard, and the test then fails.
+- **Marks and mark names:** marks in ms from `start()`. Bad names are skipped and reported in `problems`, each once, at most 20, then one line saying more were dropped. There's no `problems` field when nothing went wrong.
+- **Prompt sizes never carry text, never change the app:** a marker string put in a system prompt, a role, a list of parts and an extra key is nowhere in the record, its problems, the error log or the endpoint's answer. The messages see no write of any kind (a Proxy records every write; none), and a deeply frozen list reads fine. Lists that throw when read (a revoked Proxy, a throwing `length`, item, `role` or `content`, a `toString` that throws, a list containing itself, 100,000 messages) never throw out of `prompt()`, and the record still saves. Over HTTP, a marked request that calls `prompt()` with the marker list, a revoked Proxy or a throwing role gets the same status, headers and bytes as an unmarked one, with nothing escaping. An unmarked request reads the messages 0 times. Each was checked by breaking the code (storing the text, removing the try/catch, writing a tag on each message): the matching test then fails.
+- **Prompt sizes (`run.prompt`):** role and UTF-8 bytes per message, in order, matching `TextEncoder` (lone surrogates included); a later call replaces an earlier one; no `prompt` field without a call. Roles that aren't lowercase names are recorded as `"?"`, content that isn't a string as `bytes: null`, and something that isn't a list as no `prompt`, each with a problem naming the message by number. At most 1000 messages.
+- **`runtime`:** `"workers"` only for `navigator.userAgent === "Cloudflare-Workers"`, and absent on Node.
+- **`watchFirstChunk`:** chunks unchanged, saved once.
+- **A failed save** is logged and not thrown.
+- **MemoryStore expiry and cleanup,** including 3 hours of one write a minute never holding more than 10 records.
+
+## Known limits
+
+- **Prompt sizes: a role can still carry a little text.** `run.prompt()` keeps a role that is up to 32 lowercase letters or `_` as it is, so an app that puts text there (`ignoreallpreviousinstructions`) has it recorded and shown. Anything with a space, a capital, a digit or punctuation is recorded as `"?"`. Message content is never recorded.
+- **Saved once, at the first chunk.** Marks made after the first chunk of a streamed reply aren't in the record.
+- **`res.end(body)` doesn't mark `firstChunk`.** A reply sent only that way gets no `firstChunk` mark; only `res.write` is watched.
+- **One process only.** `MemoryStore` is for single-process servers. There's no shared store for Node yet. The Durable Object store for Workers comes with the Hono adapter.
+- **Node only, so far.** Only the `node:http` adapter exists. Express or Fastify can use it through their raw `req`/`res`, but that hasn't been tested.
+- **No cross-origin test yet.** Cross-origin APIs need the CORS preflight to allow `x-caliper-run` and `x-caliper-token`.
+- **Node 20: one end-to-end run, no test suite.** The packed package was installed into a fresh project and the [example server](https://github.com/Jia-A/caliper/tree/main/examples/node-memory) ran on Node 20.18.3 with Caliper's 3 marked visits. `npm test` runs on Node 22 only.
+- **ES module only.** Use `import`. `require("caliper-timings/node")` works on Node 22.22 but fails on Node 20.18.3 with `ERR_REQUIRE_ESM`. The type declarations were checked from an ES module with TypeScript 5.0, 5.9 and 7.0.

package/SPEC.md

@@ -0,0 +1,122 @@
+# Caliper server timings protocol, v1
+
+A server that implements this lets [Caliper](https://github.com/Jia-A/caliper) read timings from inside one marked request: how long the request waited on the database, on the model, or on any other step the app chooses to mark. Any language can implement it. `caliper-timings` is the reference implementation (JavaScript).
+
+The feature is off unless the server has a secret configured, so it can stay deployed in production, dark.
+
+## The rule: never change the app
+
+**An implementation must never change the response or error behaviour of the app's request.** A marked request must succeed or fail exactly as it would unmarked: same status, headers, bytes and errors.
+
+Every failure inside the implementation is caught there and never reaches the app's request. That includes a store that throws or rejects, a save that fails, a bad mark name, or a request it can't parse. When the record can still carry it, the failure is noted in the record's `problems` (below). Otherwise it's dropped; the reference logs it to the server's error log.
+
+A failure on Caliper's own read of the timings endpoint answers that request with 500. The app's requests are unaffected.
+
+### Measured cost
+
+The rule covers status, headers, bytes and errors, not time. This is what the reference implementation (`caliper-timings/node`) measured in time, on [`examples/node-memory`](https://github.com/Jia-A/caliper/tree/main/examples/node-memory): a single-process `node:http` app on 127.0.0.1, Node 22.22, one macOS machine. The client is `examples/node-memory/measure.mjs`, timing on its own clock from sending the request to the first body chunk arriving, one request at a time, each on a new connection.
+
+- **Marked requests: about 1.2 ms slower before the first byte.** 20 marked and 20 unmarked requests to the same server, alternating, compared in pairs: the marked one was later by median 1.2 ms (min 0.1, max 4.1). The difference is already there when the headers arrive (paired median 1.4 ms); between headers and first chunk it's 0.1 ms. What causes it hasn't been measured.
+- **`run.prompt()`: microseconds per call, measured on its own.** On Node 22.22 (Apple M2), 50 batches of 1000 calls each: a Quill-sized prompt (2 messages, 8184 bytes) took median 2.5 µs per call (min 2.4, max 4.6), and a 40-message chat (500 bytes each) median 10.0 µs (min 9.7, max 10.5). That's the call alone, not a request's time end to end, and it was measured on one machine, not on Workers. An unmarked request doesn't call it: with `run?.prompt(messages)`, `run` is `null`, so the list isn't even built (tested).
+- **Unmarked requests: no difference measured.** 20 unmarked requests to the wired server (`CALIPER_TOKEN` set) alternated with 20 to the same app before it was wired: paired difference median 0.0 ms (min −2.3, max 5.4).
+
+That is one app, one machine and 20 pairs each. Other runtimes, frameworks, stores and loads haven't been measured.
+
+## Secret
+
+The server has a secret, `CALIPER_TOKEN`. Without one (unset or empty), the feature doesn't exist: nothing is recorded, and the endpoint below answers like any unknown route (404).
+
+## Marking a request
+
+Caliper adds two headers to one request:
+
+| Header | Value |
+|---|---|
+| `x-caliper-run` | the run id: 1–64 characters of `A-Z a-z 0-9 _ -` (`^[A-Za-z0-9_-]{1,64}$`) |
+| `x-caliper-token` | the secret |
+
+The server records timings for that request only when all of these hold:
+
+- a secret is configured;
+- the run id is valid;
+- the token equals the secret. The comparison should be constant-time. The reference hashes both sides with SHA-256 and compares the digests byte by byte.
+
+In every other case the request runs exactly like an unmarked one: same status, headers and bytes, no error, nothing recorded. A bad id or token on the request is silent; the endpoint reports it later.
+
+## Marks
+
+A mark is a name and the number of milliseconds since the server's route received the request. It's a finite number; the reference records whole milliseconds from `Date.now()`.
+
+- **Names** are case-sensitive and match `^[A-Za-z][A-Za-z0-9_]{0,63}$`: a letter, then up to 63 letters, digits or `_`. A mark with any other name is skipped, never thrown, and reported in `problems`.
+- **Spans** are chosen by the app. Any pair `<name>Start` / `<name>End` is one span, called `<name>`: `dbStart` / `dbEnd` is the span `db`, and `vectorSearchStart` / `vectorSearchEnd` is the span `vectorSearch`. `<name>` must be at least one character, and the suffix must be exactly `Start` or `End`.
+- **Reserved:** `modelCallSent` (just before the model call is sent) and `firstChunk` (the first response chunk with bytes leaves the server). Caliper's report shows `firstChunk` as "server: first bytes sent" and the span between them as "model call → first bytes sent". `firstChunk` is the server's first write to the browser: if the app writes anything before the model's reply (a heartbeat, an envelope), it marks that write. Only the labels changed; the field names are as above.
+- **No span** when only one of the pair is recorded, or when `<name>End` is earlier than `<name>Start`. Caliper then shows the marks as they are and says why there's no span. It never estimates one.
+- **A step the request never reached has no mark.** Marks are never zero-filled or estimated. Marking the same name twice keeps the later value.
+- Other names are allowed, and Caliper shows them under their own name. Caliper computes spans only from pairs and the reserved model pair, never between unrelated marks.
+
+## Record
+
+```json
+{
+  "id": "3f1c…",
+  "status": 200,
+  "marks": { "dbStart": 1, "dbEnd": 301, "modelCallSent": 301, "firstChunk": 402 },
+  "runtime": "workers"
+}
+```
+
+| Field | |
+|---|---|
+| `id` | the run id from `x-caliper-run` |
+| `status` | the response status, or `null` when it isn't known (e.g. the client left before headers were sent) |
+| `marks` | name → ms, as above; may be empty |
+| `runtime` | optional, see below |
+| `prompt` | optional, added in caliper-timings 0.2.0: the sizes of the prompt's messages, see [Prompt sizes](#prompt-sizes). Leave it out when the app didn't report a prompt. |
+| `problems` | optional: a list of strings, in words, saying what the implementation skipped or couldn't do for this request. The reference writes `mark name "db-start" rejected — use a letter, then up to 63 letters, digits or _ (e.g. dbStart)`, keeps each one once and at most 20, then adds one line saying more were dropped. Leave the field out when there's nothing to report. Caliper prints each one as given, with control characters escaped and anything past 200 characters cut. A `problems` that isn't a list of strings makes Caliper reject the record. |
+
+**When it's saved:** for a streamed response, at the first chunk with bytes, so marks made after that aren't in the record. For any other response, when it finishes or the connection closes.
+
+Records should expire. The reference keeps them for 10 minutes after the last write.
+
+### Prompt sizes
+
+`prompt` is `{ "messages": [ { "role": "system", "bytes": 2310 }, { "role": "user", "bytes": 5874 } ] }`: one entry per message the app passed to the model, in order, as the app reported them just before the model call.
+
+- **`role`** is the message's role when it matches `^[a-z][a-z_]{0,31}$` (`system`, `user`, `assistant`, `tool`, …), and `"?"` otherwise, so text can never reach the record through it.
+- **`bytes`** is the length of the message's text in UTF-8, or `null` when the text wasn't a string (a list of parts, an image). It's never estimated. It's the size of the text, not of the request to the model, and not tokens.
+- **A role can still carry a little text.** A role is recorded when it matches the pattern above, so up to 32 lowercase letters or `_` that the app puts in a role (`ignoreallpreviousinstructions`) are recorded and shown as they are. Anything with a space, a capital, a digit or punctuation becomes `"?"`.
+- **Never the text.** No part of a message's content is recorded, and `problems` about the prompt name a message by its number only (`prompt message 2: …`), never by its role or text.
+- At most 1000 messages are recorded; `problems` then says how many weren't.
+- When the app passes something that isn't a list, there's no `prompt`, and `problems` says so.
+
+Older readers ignore the field, and a record without it is still valid: v1 is unchanged for servers that don't report a prompt.
+
+### `runtime`
+
+Include it only when the server has detected where it runs. Never guess or default it. The only value defined so far is `"workers"`: Cloudflare Workers, detected by `navigator.userAgent === "Cloudflare-Workers"`. On Workers, `Date.now()` only advances at I/O, so a span with only computation inside can read 0 ms. When the record says `"workers"`, Caliper prints a note saying so. Without the field, Caliper prints no runtime note.
+
+## Endpoint
+
+`GET /__caliper/timings/:id` with `Authorization: Bearer <secret>`.
+
+The checks run in this order, and the first one that fails decides the answer:
+
+| # | Check | Answer |
+|---|---|---|
+| 1 | no secret configured | **404**, like any unknown route |
+| 2 | bearer missing, not in the `Bearer <token>` form, or wrong | **401**, before the store is read |
+| 3 | id malformed | **400** |
+| 4 | no record for the id, or it expired | **404** |
+| 5 | otherwise | **200**, the record as JSON |
+
+Caliper's token pre-check depends on this order. Before any marked request it asks for the id `caliper.precheck`. The `.` makes that id invalid, so it can never name a record: with the right token the answer is 400, with a wrong one 401, and with the feature off 404.
+
+A failure reading the store (step 4) answers **500**.
+
+The reference answers 401, 400, 404 and 500 with a JSON body `{ "error": { "code": …, "message": … } }`. The codes are `UNAUTHORIZED`, `INVALID_RUN_ID`, `NOT_FOUND` and `STORE_ERROR`, and every answer carries `cache-control: no-store`. Caliper reads only the status, so the bodies are informative.
+
+The endpoint must not redirect. Caliper doesn't follow redirects, so the token can't be carried anywhere else.
+
+## Cross-origin
+
+If the page calls the API on another origin, the CORS preflight must allow the `x-caliper-run` and `x-caliper-token` request headers. Caliper hasn't been tested against a cross-origin API yet.

package/dist/index.d.ts

@@ -0,0 +1,4 @@
+export { RUN_HEADER, TOKEN_HEADER, TIMINGS_PATH, RECORD_TTL_MS, isValidRunId, isValidMarkName, isValidRole, type TimingRecord, type TimingStore, type PromptMessageSize, } from "./protocol.ts";
+export { CaliperRun, type RunOptions } from "./run.ts";
+export { MemoryStore } from "./memoryStore.ts";
+export { tokenMatches } from "./token.ts";

package/dist/index.js

@@ -0,0 +1,5 @@
+// The framework-free core: only Date.now(), Web Streams and crypto.subtle.
+export { RUN_HEADER, TOKEN_HEADER, TIMINGS_PATH, RECORD_TTL_MS, isValidRunId, isValidMarkName, isValidRole, } from "./protocol.js";
+export { CaliperRun } from "./run.js";
+export { MemoryStore } from "./memoryStore.js";
+export { tokenMatches } from "./token.js";

package/dist/memoryStore.d.ts

@@ -0,0 +1,7 @@
+import { type TimingRecord, type TimingStore } from "./protocol.ts";
+export declare class MemoryStore implements TimingStore {
+    #private;
+    put(id: string, record: TimingRecord): Promise<void>;
+    get(id: string): Promise<TimingRecord | null>;
+    get size(): number;
+}

package/dist/memoryStore.js

@@ -0,0 +1,31 @@
+import { RECORD_TTL_MS } from "./protocol.js";
+// Single-process servers only: on Workers each isolate has its own Map (and so does each process
+// behind a load balancer), so a read can land where the write never happened.
+// Cleanup: every put deletes every expired record first, and a get that finds an expired record
+// deletes it. So after a put, the Map holds only records written in the last RECORD_TTL_MS. If
+// writes stop, what's left stays until the next put or get: a bounded leftover, not a leak.
+export class MemoryStore {
+    #records = new Map();
+    async put(id, record) {
+        const now = Date.now();
+        for (const [key, entry] of this.#records) {
+            if (entry.expiresAt <= now)
+                this.#records.delete(key);
+        }
+        this.#records.set(id, { record, expiresAt: now + RECORD_TTL_MS });
+    }
+    async get(id) {
+        const entry = this.#records.get(id);
+        if (!entry)
+            return null;
+        if (entry.expiresAt <= Date.now()) {
+            this.#records.delete(id);
+            return null;
+        }
+        return entry.record;
+    }
+    // Records held, including expired ones not deleted yet.
+    get size() {
+        return this.#records.size;
+    }
+}

package/dist/node.d.ts

@@ -0,0 +1,13 @@
+import type { IncomingMessage, ServerResponse } from "node:http";
+import { type TimingStore } from "./protocol.ts";
+import { CaliperRun } from "./run.ts";
+export * from "./index.ts";
+export type TimingsOptions = {
+    token: string | undefined;
+    store: TimingStore;
+};
+export type Timings = {
+    handle(req: IncomingMessage, res: ServerResponse): Promise<boolean>;
+    start(req: IncomingMessage, res: ServerResponse): Promise<CaliperRun | null>;
+};
+export declare function createTimings({ token, store }: TimingsOptions): Timings;

package/dist/node.js

@@ -0,0 +1,131 @@
+import { isValidRunId, RUN_HEADER, TIMINGS_PATH, TOKEN_HEADER } from "./protocol.js";
+import { CaliperRun, hasBytes } from "./run.js";
+import { tokenMatches } from "./token.js";
+export * from "./index.js";
+const header = (req, name) => {
+    const value = req.headers[name];
+    return typeof value === "string" ? value : undefined;
+};
+function sendJson(res, status, body) {
+    res.writeHead(status, { "content-type": "application/json", "cache-control": "no-store" });
+    res.end(JSON.stringify(body));
+    return true;
+}
+export function createTimings({ token, store }) {
+    const secret = token || undefined;
+    return {
+        // GET /__caliper/timings/:id, checked in SPEC.md's order. Returns false (and leaves the
+        // response alone) for any other request, and for every request when no secret is set, so
+        // the app answers it like any unknown route.
+        async handle(req, res) {
+            if (!secret || req.method !== "GET")
+                return false;
+            let path;
+            try {
+                path = new URL(req.url ?? "/", "http://localhost").pathname;
+            }
+            catch {
+                return false; // a URL that doesn't parse is the app's to answer
+            }
+            if (!path.startsWith(TIMINGS_PATH))
+                return false;
+            // Checked before anything else, so an unauthenticated call never reads the store.
+            const bearer = /^Bearer (.+)$/.exec(header(req, "authorization") ?? "")?.[1];
+            if (bearer === undefined || !(await tokenMatches(bearer, secret))) {
+                return sendJson(res, 401, { error: { code: "UNAUTHORIZED", message: "Missing or invalid Caliper token" } });
+            }
+            let id = "";
+            try {
+                id = decodeURIComponent(path.slice(TIMINGS_PATH.length));
+            }
+            catch {
+                // Not valid percent-encoding: rejected as malformed below.
+            }
+            if (!isValidRunId(id)) {
+                return sendJson(res, 400, {
+                    error: { code: "INVALID_RUN_ID", message: "Run id must be 1-64 characters of A-Z, a-z, 0-9, _ or -" },
+                });
+            }
+            let record;
+            try {
+                record = await store.get(id);
+            }
+            catch (err) {
+                logFailure("Caliper timing record not read", err);
+                return sendJson(res, 500, { error: { code: "STORE_ERROR", message: "The timing store failed; see the server log" } });
+            }
+            if (!record)
+                return sendJson(res, 404, { error: { code: "NOT_FOUND", message: "No timing record for this run id" } });
+            return sendJson(res, 200, record);
+        },
+        // A CaliperRun when the request carries a valid x-caliper-run and the right x-caliper-token,
+        // otherwise null and the request and response are left exactly as they were. A malformed id
+        // is ignored here; the endpoint rejects it with 400 when Caliper asks for it. Never throws:
+        // if anything fails, the request runs unmarked.
+        async start(req, res) {
+            try {
+                const arrived = Date.now();
+                const id = header(req, RUN_HEADER);
+                if (id === undefined || !secret)
+                    return null;
+                const given = header(req, TOKEN_HEADER);
+                if (given === undefined || !isValidRunId(id))
+                    return null;
+                if (!(await tokenMatches(given, secret)))
+                    return null;
+                const run = new CaliperRun(id, store, { start: arrived });
+                watchResponse(run, res);
+                return run;
+            }
+            catch (err) {
+                logFailure("Caliper timings off for this request", err);
+                return null;
+            }
+        },
+    };
+}
+function logFailure(what, err) {
+    try {
+        console.error(what, err);
+    }
+    catch {
+        // Not even the log: drop it.
+    }
+}
+// The record is saved once: at the first res.write with bytes (a streamed reply: firstChunk is
+// marked then), or else when the response closes (finished, or the client left). Every write
+// goes through unchanged, and whatever the original write does (returns, throws) is what the app
+// sees; after the first, res.write is the original again. The wrapper's own part can't throw.
+function watchResponse(run, res) {
+    let saved = false;
+    const write = res.write;
+    res.write = function (...args) {
+        // Backstop for the never-change-the-app rule (SPEC.md). Nothing inside can throw today;
+        // this protects future edits. No test can trigger it.
+        try {
+            if (!saved && hasBytes(args[0])) {
+                saved = true;
+                res.write = write;
+                run.status = res.statusCode;
+                run.mark("firstChunk");
+                void run.save();
+            }
+        }
+        catch (err) {
+            logFailure("Caliper timing record not saved", err);
+        }
+        return write.apply(this, args);
+    };
+    res.once("close", () => {
+        try {
+            if (saved)
+                return;
+            saved = true;
+            run.status = res.headersSent ? res.statusCode : null;
+            void run.save();
+        }
+        catch (err) {
+            logFailure("Caliper timing record not saved", err);
+        }
+    });
+}

package/dist/protocol.d.ts

@@ -0,0 +1,25 @@
+export declare const RUN_HEADER = "x-caliper-run";
+export declare const TOKEN_HEADER = "x-caliper-token";
+export declare const TIMINGS_PATH = "/__caliper/timings/";
+export type TimingRecord = {
+    id: string;
+    runtime?: "workers";
+    status: number | null;
+    marks: Record<string, number>;
+    problems?: string[];
+    prompt?: {
+        messages: PromptMessageSize[];
+    };
+};
+export type PromptMessageSize = {
+    role: string;
+    bytes: number | null;
+};
+export interface TimingStore {
+    put(id: string, record: TimingRecord): Promise<void>;
+    get(id: string): Promise<TimingRecord | null>;
+}
+export declare const RECORD_TTL_MS: number;
+export declare const isValidRunId: (id: string) => boolean;
+export declare const isValidMarkName: (name: unknown) => name is string;
+export declare const isValidRole: (role: unknown) => role is string;

package/dist/protocol.js

@@ -0,0 +1,15 @@
+// The wire format of SPEC.md: headers, ids, mark names, the record and where it's read.
+export const RUN_HEADER = "x-caliper-run";
+export const TOKEN_HEADER = "x-caliper-token";
+export const TIMINGS_PATH = "/__caliper/timings/";
+export const RECORD_TTL_MS = 10 * 60 * 1000;
+const RUN_ID = /^[A-Za-z0-9_-]{1,64}$/;
+export const isValidRunId = (id) => RUN_ID.test(id);
+// Case-sensitive. <name>Start / <name>End is a span called <name>; modelCallSent and firstChunk
+// are reserved for the model call.
+const MARK_NAME = /^[A-Za-z][A-Za-z0-9_]{0,63}$/;
+export const isValidMarkName = (name) => typeof name === "string" && MARK_NAME.test(name);
+// Lowercase letters and _, up to 32: system, user, assistant, tool, function_call. Anything else
+// could be text, so it's never recorded.
+const ROLE = /^[a-z][a-z_]{0,31}$/;
+export const isValidRole = (role) => typeof role === "string" && ROLE.test(role);

package/dist/run.d.ts

@@ -0,0 +1,17 @@
+import { type TimingStore } from "./protocol.ts";
+export declare const hasBytes: (chunk: unknown) => boolean;
+export type RunOptions = {
+    start?: number;
+    defer?: (p: Promise<void>) => void;
+};
+export declare class CaliperRun {
+    #private;
+    readonly id: string;
+    status: number | null;
+    constructor(id: string, store: TimingStore, { start, defer }?: RunOptions);
+    mark(name: string): void;
+    prompt(messages: unknown): void;
+    watchFirstChunk<T>(stream: ReadableStream<T>): ReadableStream<T>;
+    save(): Promise<void>;
+    get savesOnStream(): boolean;
+}

package/dist/run.js

@@ -0,0 +1,145 @@
+import { isValidMarkName, isValidRole } from "./protocol.js";
+// Cloudflare documents navigator.userAgent === "Cloudflare-Workers" as the way
+// to tell code is running on Workers (global_navigator flag, on by default
+// since compatibility date 2022-03-21):
+// https://developers.cloudflare.com/workers/runtime-apis/web-standards/
+const isWorkers = () => globalThis.navigator?.userAgent === "Cloudflare-Workers";
+// A chunk with no bytes (an empty write to flush headers) isn't the first chunk.
+export const hasBytes = (chunk) => typeof chunk === "string" ? chunk.length > 0
+    : ArrayBuffer.isView(chunk) ? chunk.byteLength > 0
+        : chunk !== undefined && chunk !== null;
+const MAX_PROBLEMS = 20;
+const MAX_PROMPT_MESSAGES = 1000;
+const utf8 = new TextEncoder();
+// Logged, never thrown: nothing the wrapper does may reach the app's request (SPEC.md).
+function logFailure(err) {
+    try {
+        console.error("Caliper timing record not saved", err);
+    }
+    catch {
+        // Not even the log: drop it.
+    }
+}
+// One marked request. On Workers, Date.now() is frozen while code runs and only
+// advances at I/O, so a span with only computation inside reads 0 ms there.
+// No method throws or rejects: a marked request succeeds or fails exactly as unmarked.
+export class CaliperRun {
+    id;
+    status = null;
+    #start;
+    #marks = {};
+    #problems = [];
+    #prompt = null;
+    #store;
+    #defer;
+    #watchingStream = false;
+    constructor(id, store, { start = Date.now(), defer = () => { } } = {}) {
+        this.id = id;
+        this.#store = store;
+        this.#start = start;
+        this.#defer = defer;
+    }
+    // A name SPEC.md doesn't allow is skipped and reported in the record's problems, never thrown.
+    mark(name) {
+        if (!isValidMarkName(name)) {
+            this.#problem(`mark name ${JSON.stringify(name)} rejected — use a letter, then up to 63 letters, digits or _ (e.g. dbStart)`);
+            return;
+        }
+        this.#marks[name] = Date.now() - this.#start;
+    }
+    // Each message's role and UTF-8 size, never its text; reads the messages and never changes them.
+    // Call it just before the model call. A later call replaces an earlier one. Never throws.
+    prompt(messages) {
+        try {
+            if (!Array.isArray(messages)) {
+                this.#problem("prompt not recorded: run.prompt() takes a list of { role, content } messages");
+                return;
+            }
+            const sizes = [];
+            const kept = Math.min(messages.length, MAX_PROMPT_MESSAGES);
+            for (let i = 0; i < kept; i++)
+                sizes.push(this.#size(messages[i], i + 1));
+            if (messages.length > kept) {
+                this.#problem(`prompt: ${messages.length - kept} more messages not recorded (only the first ${MAX_PROMPT_MESSAGES} are kept)`);
+            }
+            this.#prompt = sizes;
+        }
+        catch {
+            this.#problem("prompt not recorded: reading the messages failed");
+        }
+    }
+    // Problems name the message by number only: its role or text could be anything.
+    #size(message, n) {
+        let role;
+        let content;
+        try {
+            ({ role, content } = (message ?? {}));
+        }
+        catch {
+            this.#problem(`prompt message ${n}: reading it failed; recorded as "?" with no size`);
+            return { role: "?", bytes: null };
+        }
+        if (!isValidRole(role)) {
+            this.#problem(`prompt message ${n}: role isn't a lowercase name of up to 32 letters or _ (e.g. system, user, assistant); recorded as "?"`);
+        }
+        if (typeof content !== "string")
+            this.#problem(`prompt message ${n}: content isn't a string; size not measured`);
+        return { role: isValidRole(role) ? role : "?", bytes: typeof content === "string" ? utf8.encode(content).length : null };
+    }
+    // Each problem once; after MAX_PROBLEMS, one line says the rest were dropped.
+    #problem(text) {
+        if (this.#problems.includes(text) || this.#problems.length > MAX_PROBLEMS)
+            return;
+        this.#problems.push(this.#problems.length < MAX_PROBLEMS ? text : `more problems not recorded (only the first ${MAX_PROBLEMS} are kept)`);
+    }
+    // Passes every chunk through unchanged (same bytes, same chunk boundaries);
+    // it only notes when the first one with bytes arrives, then saves the record.
+    watchFirstChunk(stream) {
+        this.#watchingStream = true;
+        let seen = false;
+        return stream.pipeThrough(new TransformStream({
+            transform: (chunk, controller) => {
+                if (!seen && hasBytes(chunk)) {
+                    seen = true;
+                    this.mark("firstChunk");
+                    void this.save();
+                }
+                controller.enqueue(chunk);
+            },
+            flush: () => {
+                if (!seen)
+                    void this.save();
+            },
+        }));
+    }
+    // Never throws and never rejects, whatever the store or defer does: a failure is logged and
+    // the record is dropped (it can't carry its own failure).
+    save() {
+        let saved;
+        try {
+            const record = {
+                id: this.id,
+                ...(isWorkers() && { runtime: "workers" }),
+                status: this.status,
+                marks: { ...this.#marks },
+                ...(this.#problems.length && { problems: [...this.#problems] }),
+                ...(this.#prompt && { prompt: { messages: this.#prompt.map((m) => ({ ...m })) } }),
+            };
+            saved = Promise.resolve(this.#store.put(this.id, record)).catch(logFailure);
+        }
+        catch (err) {
+            logFailure(err);
+            saved = Promise.resolve();
+        }
+        try {
+            this.#defer(saved);
+        }
+        catch (err) {
+            logFailure(err);
+        }
+        return saved;
+    }
+    get savesOnStream() {
+        return this.#watchingStream;
+    }
+}

package/dist/token.d.ts

@@ -0,0 +1 @@
+export declare function tokenMatches(given: string, expected: string): Promise<boolean>;

package/dist/token.js

@@ -0,0 +1,13 @@
+// Constant-time check of a Caliper token against the CALIPER_TOKEN secret.
+// Both sides are hashed first so the comparison always runs over 32 bytes,
+// whatever the length of the given token.
+export async function tokenMatches(given, expected) {
+    const enc = new TextEncoder();
+    const [a, b] = await Promise.all([given, expected].map((s) => crypto.subtle.digest("SHA-256", enc.encode(s))));
+    const x = new Uint8Array(a);
+    const y = new Uint8Array(b);
+    let diff = 0;
+    for (let i = 0; i < x.length; i++)
+        diff |= x[i] ^ y[i];
+    return diff === 0;
+}

package/package.json

@@ -1,6 +1,51 @@
 {
   "name": "caliper-timings",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.2.0",
+  "description": "Server side of Caliper's server timings: records when a marked request reaches each step, for caliper-ai to read back.",
+  "keywords": [
+    "caliper",
+    "ai",
+    "llm",
+    "performance",
+    "latency",
+    "server-timing"
+  ],
+  "author": "Jiya Agrawal",
+  "license": "MIT",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/Jia-A/caliper.git",
+    "directory": "packages/timings"
+  },
+  "homepage": "https://github.com/Jia-A/caliper#readme",
+  "type": "module",
+  "exports": {
+    ".": {
+      "types": "./dist/index.d.ts",
+      "default": "./dist/index.js"
+    },
+    "./node": {
+      "types": "./dist/node.d.ts",
+      "default": "./dist/node.js"
+    }
+  },
+  "files": [
+    "dist/",
+    "README.md",
+    "SPEC.md",
+    "CHANGELOG.md",
+    "LICENSE"
+  ],
+  "engines": {
+    "node": ">=20"
+  },
+  "scripts": {
+    "build": "tsc -p tsconfig.build.json",
+    "typecheck": "tsc --noEmit",
+    "test": "node --test --test-timeout=20000 test/*.test.ts"
+  },
+  "devDependencies": {
+    "@types/node": "^20.19.43",
+    "typescript": "^7.0.2"
+  }
+}