caliper-timings 0.1.0 → 0.2.0

package/CHANGELOG.md

@@ -1,5 +1,16 @@
 # Changelog
 
+## 0.2.0 — 2026-11-12
+
+### Added
+
+- **`run.prompt(messages)`: prompt sizes.** Records each message's role and the size of its text in bytes (UTF-8), in order, in the record's new optional `prompt` field (`{ "messages": [{ "role": "system", "bytes": 2310 }, …] }`). Never the text: a role that isn't a lowercase name of up to 32 letters or `_` is recorded as `"?"`, content that isn't a string gets `bytes: null` (never estimated), and problems name the message by number only. At most 1000 messages. Never throws.
+- **SPEC v1: optional `prompt` field** ([Prompt sizes](SPEC.md#prompt-sizes)). Not breaking: a record without it is unchanged. caliper-ai 0.4.0's record reader (from the release commit) was run once on a record with `prompt`: it read the marks and ignored the field.
+- `isValidRole` and the `PromptMessageSize` type are exported.
+- **Known limit:** a role of up to 32 lowercase letters or `_` is recorded as it is, so text an app puts in a role in that form is recorded too. Message content never is.
+- **Tests that `run.prompt()` never carries text or changes the app** (a marker in every part of a prompt; a Proxy that records writes; lists that throw when read; byte-identical responses over HTTP; no reads when unmarked), each checked by breaking the code.
+- **Measured cost of `run.prompt()`:** median 2.5 µs per call for a Quill-sized prompt (8184 bytes), 10.0 µs for 40 messages, on Node 22.22 (Apple M2).
+
 ## 0.1.0 — 2026-11-04
 
 ### Added

package/README.md

@@ -52,6 +52,8 @@ createServer(async (req, res) => {
     run?.mark("dbStart");
     const row = await db.query(/* … */);
     run?.mark("dbEnd");
+    const messages = [{ role: "system", content: SYSTEM }, { role: "user", content: row.text }];
+    run?.prompt(messages); // each message's role and size in bytes, never its text
     run?.mark("modelCallSent");
     res.writeHead(200, { "content-type": "text/event-stream" });
     for await (const chunk of model.stream(/* … */)) res.write(chunk); // the first write with bytes: firstChunk
@@ -65,6 +67,7 @@ createServer(async (req, res) => {
 - **`handle(req, res)`** answers `GET /__caliper/timings/:id` in SPEC.md's check order and returns `true`. For any other request it returns `false` and doesn't touch the response. With no token set it returns `false` for every request, so your own 404 answers.
 - **`start(req, res)`** returns a `CaliperRun` only when the request carries a valid `x-caliper-run` and the right `x-caliper-token`. Otherwise it returns `null` and the request and response are left exactly as they were. Call it first thing in the route: marks are measured from the moment `start()` was called.
 - **`run.mark(name)`** records ms since then. Name spans with `<name>Start` / `<name>End` (`dbStart` / `dbEnd`); Caliper shows each pair as a span called `<name>`. `modelCallSent` and `firstChunk` are reserved. A name outside `^[A-Za-z][A-Za-z0-9_]{0,63}$` is skipped, not thrown, and Caliper prints it under "problems reported by the server".
+- **`run.prompt(messages)`** records each message's role and its size in bytes (UTF-8), in order, for Caliper's prompt sizes. Pass the same list you send to the model, as `{ role, content }` objects, just before the model call. With a separate system text (Anthropic's API), pass it as the first message. It never records the text. A role that isn't a lowercase name (`system`, `user`, `assistant`, …) is recorded as `"?"`, and content that isn't a string gets no size (`null`), with a problem saying which message, by number. It reads the list and never changes it. A later call replaces an earlier one. See [SPEC.md](SPEC.md#prompt-sizes).
 - **When the record is saved:** at the first `res.write` with bytes, with `firstChunk` marked then. Otherwise when the response closes, whether it finished or the client left. `status` is `res.statusCode`, or `null` if no headers were sent. Every write goes through unchanged.
 
 ### It never changes your app
@@ -88,11 +91,11 @@ A marked request succeeds or fails exactly as it would unmarked; that's the prot
 
 The core works in any runtime with Web Streams and `crypto.subtle`. It exports:
 
-- `CaliperRun`, with `mark`, `save`, and `watchFirstChunk(stream)` for frameworks that return a `ReadableStream`;
+- `CaliperRun`, with `mark`, `prompt`, `save`, and `watchFirstChunk(stream)` for frameworks that return a `ReadableStream`;
 - `MemoryStore` and the `TimingStore` interface;
-- `tokenMatches`, `isValidRunId`, `isValidMarkName`;
+- `tokenMatches`, `isValidRunId`, `isValidMarkName`, `isValidRole`;
 - the header and path constants;
-- the `TimingRecord` type.
+- the `TimingRecord` and `PromptMessageSize` types.
 
 There's no framework adapter other than Node yet.
 
@@ -101,6 +104,7 @@ There's no framework adapter other than Node yet.
 The rule covers status, headers, bytes and errors, not time. This is what the reference implementation (`caliper-timings/node`) measured in time, on [`examples/node-memory`](https://github.com/Jia-A/caliper/tree/main/examples/node-memory): a single-process `node:http` app on 127.0.0.1, Node 22.22, one macOS machine. The client is `examples/node-memory/measure.mjs`, timing on its own clock from sending the request to the first body chunk arriving, one request at a time, each on a new connection.
 
 - **Marked requests: about 1.2 ms slower before the first byte.** 20 marked and 20 unmarked requests to the same server, alternating, compared in pairs: the marked one was later by median 1.2 ms (min 0.1, max 4.1). The difference is already there when the headers arrive (paired median 1.4 ms); between headers and first chunk it's 0.1 ms. What causes it hasn't been measured.
+- **`run.prompt()`: microseconds per call, measured on its own.** On Node 22.22 (Apple M2), 50 batches of 1000 calls each: a Quill-sized prompt (2 messages, 8184 bytes) took median 2.5 µs per call (min 2.4, max 4.6), and a 40-message chat (500 bytes each) median 10.0 µs (min 9.7, max 10.5). That's the call alone, not a request's time end to end, and it was measured on one machine, not on Workers. An unmarked request doesn't call it: with `run?.prompt(messages)`, `run` is `null`, so the list isn't even built (tested).
 - **Unmarked requests: no difference measured.** 20 unmarked requests to the wired server (`CALIPER_TOKEN` set) alternated with 20 to the same app before it was wired: paired difference median 0.0 ms (min −2.3, max 5.4).
 
 That is one app, one machine and 20 pairs each. Other runtimes, frameworks, stores and loads haven't been measured.
@@ -114,6 +118,8 @@ That is one app, one machine and 20 pairs each. Other runtimes, frameworks, stor
 - **When the record is saved:** at the first write with bytes (an empty write doesn't count), at close for `res.end(body)` replies, and at close with `status: null` when the client leaves before headers.
 - **Never changing the app:** with a store whose `put` throws, or one whose `put` rejects, the marked response is byte-identical to the unmarked one and nothing escapes as an unhandled error. With a broken `get`, the endpoint answers 500 and app requests are untouched. A URL that doesn't parse is left to the app. A `save()` with a throwing store or `defer` neither throws nor rejects. Each of these was checked by removing the guard, and the test then fails.
 - **Marks and mark names:** marks in ms from `start()`. Bad names are skipped and reported in `problems`, each once, at most 20, then one line saying more were dropped. There's no `problems` field when nothing went wrong.
+- **Prompt sizes never carry text, never change the app:** a marker string put in a system prompt, a role, a list of parts and an extra key is nowhere in the record, its problems, the error log or the endpoint's answer. The messages see no write of any kind (a Proxy records every write; none), and a deeply frozen list reads fine. Lists that throw when read (a revoked Proxy, a throwing `length`, item, `role` or `content`, a `toString` that throws, a list containing itself, 100,000 messages) never throw out of `prompt()`, and the record still saves. Over HTTP, a marked request that calls `prompt()` with the marker list, a revoked Proxy or a throwing role gets the same status, headers and bytes as an unmarked one, with nothing escaping. An unmarked request reads the messages 0 times. Each was checked by breaking the code (storing the text, removing the try/catch, writing a tag on each message): the matching test then fails.
+- **Prompt sizes (`run.prompt`):** role and UTF-8 bytes per message, in order, matching `TextEncoder` (lone surrogates included); a later call replaces an earlier one; no `prompt` field without a call. Roles that aren't lowercase names are recorded as `"?"`, content that isn't a string as `bytes: null`, and something that isn't a list as no `prompt`, each with a problem naming the message by number. At most 1000 messages.
 - **`runtime`:** `"workers"` only for `navigator.userAgent === "Cloudflare-Workers"`, and absent on Node.
 - **`watchFirstChunk`:** chunks unchanged, saved once.
 - **A failed save** is logged and not thrown.
@@ -121,6 +127,7 @@ That is one app, one machine and 20 pairs each. Other runtimes, frameworks, stor
 
 ## Known limits
 
+- **Prompt sizes: a role can still carry a little text.** `run.prompt()` keeps a role that is up to 32 lowercase letters or `_` as it is, so an app that puts text there (`ignoreallpreviousinstructions`) has it recorded and shown. Anything with a space, a capital, a digit or punctuation is recorded as `"?"`. Message content is never recorded.
 - **Saved once, at the first chunk.** Marks made after the first chunk of a streamed reply aren't in the record.
 - **`res.end(body)` doesn't mark `firstChunk`.** A reply sent only that way gets no `firstChunk` mark; only `res.write` is watched.
 - **One process only.** `MemoryStore` is for single-process servers. There's no shared store for Node yet. The Durable Object store for Workers comes with the Hono adapter.

package/SPEC.md

@@ -17,6 +17,7 @@ A failure on Caliper's own read of the timings endpoint answers that request wit
 The rule covers status, headers, bytes and errors, not time. This is what the reference implementation (`caliper-timings/node`) measured in time, on [`examples/node-memory`](https://github.com/Jia-A/caliper/tree/main/examples/node-memory): a single-process `node:http` app on 127.0.0.1, Node 22.22, one macOS machine. The client is `examples/node-memory/measure.mjs`, timing on its own clock from sending the request to the first body chunk arriving, one request at a time, each on a new connection.
 
 - **Marked requests: about 1.2 ms slower before the first byte.** 20 marked and 20 unmarked requests to the same server, alternating, compared in pairs: the marked one was later by median 1.2 ms (min 0.1, max 4.1). The difference is already there when the headers arrive (paired median 1.4 ms); between headers and first chunk it's 0.1 ms. What causes it hasn't been measured.
+- **`run.prompt()`: microseconds per call, measured on its own.** On Node 22.22 (Apple M2), 50 batches of 1000 calls each: a Quill-sized prompt (2 messages, 8184 bytes) took median 2.5 µs per call (min 2.4, max 4.6), and a 40-message chat (500 bytes each) median 10.0 µs (min 9.7, max 10.5). That's the call alone, not a request's time end to end, and it was measured on one machine, not on Workers. An unmarked request doesn't call it: with `run?.prompt(messages)`, `run` is `null`, so the list isn't even built (tested).
 - **Unmarked requests: no difference measured.** 20 unmarked requests to the wired server (`CALIPER_TOKEN` set) alternated with 20 to the same app before it was wired: paired difference median 0.0 ms (min −2.3, max 5.4).
 
 That is one app, one machine and 20 pairs each. Other runtimes, frameworks, stores and loads haven't been measured.
@@ -48,7 +49,7 @@ A mark is a name and the number of milliseconds since the server's route receive
 
 - **Names** are case-sensitive and match `^[A-Za-z][A-Za-z0-9_]{0,63}$`: a letter, then up to 63 letters, digits or `_`. A mark with any other name is skipped, never thrown, and reported in `problems`.
 - **Spans** are chosen by the app. Any pair `<name>Start` / `<name>End` is one span, called `<name>`: `dbStart` / `dbEnd` is the span `db`, and `vectorSearchStart` / `vectorSearchEnd` is the span `vectorSearch`. `<name>` must be at least one character, and the suffix must be exactly `Start` or `End`.
-- **Reserved:** `modelCallSent` (just before the model call is sent) and `firstChunk` (the first response chunk with bytes leaves the server). Caliper shows the span between them as "model: sent → first chunk".
+- **Reserved:** `modelCallSent` (just before the model call is sent) and `firstChunk` (the first response chunk with bytes leaves the server). Caliper's report shows `firstChunk` as "server: first bytes sent" and the span between them as "model call → first bytes sent". `firstChunk` is the server's first write to the browser: if the app writes anything before the model's reply (a heartbeat, an envelope), it marks that write. Only the labels changed; the field names are as above.
 - **No span** when only one of the pair is recorded, or when `<name>End` is earlier than `<name>Start`. Caliper then shows the marks as they are and says why there's no span. It never estimates one.
 - **A step the request never reached has no mark.** Marks are never zero-filled or estimated. Marking the same name twice keeps the later value.
 - Other names are allowed, and Caliper shows them under their own name. Caliper computes spans only from pairs and the reserved model pair, never between unrelated marks.
@@ -70,12 +71,26 @@ A mark is a name and the number of milliseconds since the server's route receive
 | `status` | the response status, or `null` when it isn't known (e.g. the client left before headers were sent) |
 | `marks` | name → ms, as above; may be empty |
 | `runtime` | optional, see below |
+| `prompt` | optional, added in caliper-timings 0.2.0: the sizes of the prompt's messages, see [Prompt sizes](#prompt-sizes). Leave it out when the app didn't report a prompt. |
 | `problems` | optional: a list of strings, in words, saying what the implementation skipped or couldn't do for this request. The reference writes `mark name "db-start" rejected — use a letter, then up to 63 letters, digits or _ (e.g. dbStart)`, keeps each one once and at most 20, then adds one line saying more were dropped. Leave the field out when there's nothing to report. Caliper prints each one as given, with control characters escaped and anything past 200 characters cut. A `problems` that isn't a list of strings makes Caliper reject the record. |
 
 **When it's saved:** for a streamed response, at the first chunk with bytes, so marks made after that aren't in the record. For any other response, when it finishes or the connection closes.
 
 Records should expire. The reference keeps them for 10 minutes after the last write.
 
+### Prompt sizes
+
+`prompt` is `{ "messages": [ { "role": "system", "bytes": 2310 }, { "role": "user", "bytes": 5874 } ] }`: one entry per message the app passed to the model, in order, as the app reported them just before the model call.
+
+- **`role`** is the message's role when it matches `^[a-z][a-z_]{0,31}$` (`system`, `user`, `assistant`, `tool`, …), and `"?"` otherwise, so text can never reach the record through it.
+- **`bytes`** is the length of the message's text in UTF-8, or `null` when the text wasn't a string (a list of parts, an image). It's never estimated. It's the size of the text, not of the request to the model, and not tokens.
+- **A role can still carry a little text.** A role is recorded when it matches the pattern above, so up to 32 lowercase letters or `_` that the app puts in a role (`ignoreallpreviousinstructions`) are recorded and shown as they are. Anything with a space, a capital, a digit or punctuation becomes `"?"`.
+- **Never the text.** No part of a message's content is recorded, and `problems` about the prompt name a message by its number only (`prompt message 2: …`), never by its role or text.
+- At most 1000 messages are recorded; `problems` then says how many weren't.
+- When the app passes something that isn't a list, there's no `prompt`, and `problems` says so.
+
+Older readers ignore the field, and a record without it is still valid: v1 is unchanged for servers that don't report a prompt.
+
 ### `runtime`
 
 Include it only when the server has detected where it runs. Never guess or default it. The only value defined so far is `"workers"`: Cloudflare Workers, detected by `navigator.userAgent === "Cloudflare-Workers"`. On Workers, `Date.now()` only advances at I/O, so a span with only computation inside can read 0 ms. When the record says `"workers"`, Caliper prints a note saying so. Without the field, Caliper prints no runtime note.

package/dist/index.d.ts

@@ -1,4 +1,4 @@
-export { RUN_HEADER, TOKEN_HEADER, TIMINGS_PATH, RECORD_TTL_MS, isValidRunId, isValidMarkName, type TimingRecord, type TimingStore, } from "./protocol.ts";
+export { RUN_HEADER, TOKEN_HEADER, TIMINGS_PATH, RECORD_TTL_MS, isValidRunId, isValidMarkName, isValidRole, type TimingRecord, type TimingStore, type PromptMessageSize, } from "./protocol.ts";
 export { CaliperRun, type RunOptions } from "./run.ts";
 export { MemoryStore } from "./memoryStore.ts";
 export { tokenMatches } from "./token.ts";

package/dist/index.js

@@ -1,5 +1,5 @@
 // The framework-free core: only Date.now(), Web Streams and crypto.subtle.
-export { RUN_HEADER, TOKEN_HEADER, TIMINGS_PATH, RECORD_TTL_MS, isValidRunId, isValidMarkName, } from "./protocol.js";
+export { RUN_HEADER, TOKEN_HEADER, TIMINGS_PATH, RECORD_TTL_MS, isValidRunId, isValidMarkName, isValidRole, } from "./protocol.js";
 export { CaliperRun } from "./run.js";
 export { MemoryStore } from "./memoryStore.js";
 export { tokenMatches } from "./token.js";

package/dist/protocol.d.ts

@@ -7,6 +7,13 @@ export type TimingRecord = {
     status: number | null;
     marks: Record<string, number>;
     problems?: string[];
+    prompt?: {
+        messages: PromptMessageSize[];
+    };
+};
+export type PromptMessageSize = {
+    role: string;
+    bytes: number | null;
 };
 export interface TimingStore {
     put(id: string, record: TimingRecord): Promise<void>;
@@ -15,3 +22,4 @@ export interface TimingStore {
 export declare const RECORD_TTL_MS: number;
 export declare const isValidRunId: (id: string) => boolean;
 export declare const isValidMarkName: (name: unknown) => name is string;
+export declare const isValidRole: (role: unknown) => role is string;

package/dist/protocol.js

@@ -9,3 +9,7 @@ export const isValidRunId = (id) => RUN_ID.test(id);
 // are reserved for the model call.
 const MARK_NAME = /^[A-Za-z][A-Za-z0-9_]{0,63}$/;
 export const isValidMarkName = (name) => typeof name === "string" && MARK_NAME.test(name);
+// Lowercase letters and _, up to 32: system, user, assistant, tool, function_call. Anything else
+// could be text, so it's never recorded.
+const ROLE = /^[a-z][a-z_]{0,31}$/;
+export const isValidRole = (role) => typeof role === "string" && ROLE.test(role);

package/dist/run.d.ts

@@ -10,6 +10,7 @@ export declare class CaliperRun {
     status: number | null;
     constructor(id: string, store: TimingStore, { start, defer }?: RunOptions);
     mark(name: string): void;
+    prompt(messages: unknown): void;
     watchFirstChunk<T>(stream: ReadableStream<T>): ReadableStream<T>;
     save(): Promise<void>;
     get savesOnStream(): boolean;

package/dist/run.js

@@ -1,4 +1,4 @@
-import { isValidMarkName } from "./protocol.js";
+import { isValidMarkName, isValidRole } from "./protocol.js";
 // Cloudflare documents navigator.userAgent === "Cloudflare-Workers" as the way
 // to tell code is running on Workers (global_navigator flag, on by default
 // since compatibility date 2022-03-21):
@@ -9,6 +9,8 @@ export const hasBytes = (chunk) => typeof chunk === "string" ? chunk.length > 0
     : ArrayBuffer.isView(chunk) ? chunk.byteLength > 0
         : chunk !== undefined && chunk !== null;
 const MAX_PROBLEMS = 20;
+const MAX_PROMPT_MESSAGES = 1000;
+const utf8 = new TextEncoder();
 // Logged, never thrown: nothing the wrapper does may reach the app's request (SPEC.md).
 function logFailure(err) {
     try {
@@ -27,6 +29,7 @@ export class CaliperRun {
     #start;
     #marks = {};
     #problems = [];
+    #prompt = null;
     #store;
     #defer;
     #watchingStream = false;
@@ -44,6 +47,45 @@ export class CaliperRun {
         }
         this.#marks[name] = Date.now() - this.#start;
     }
+    // Each message's role and UTF-8 size, never its text; reads the messages and never changes them.
+    // Call it just before the model call. A later call replaces an earlier one. Never throws.
+    prompt(messages) {
+        try {
+            if (!Array.isArray(messages)) {
+                this.#problem("prompt not recorded: run.prompt() takes a list of { role, content } messages");
+                return;
+            }
+            const sizes = [];
+            const kept = Math.min(messages.length, MAX_PROMPT_MESSAGES);
+            for (let i = 0; i < kept; i++)
+                sizes.push(this.#size(messages[i], i + 1));
+            if (messages.length > kept) {
+                this.#problem(`prompt: ${messages.length - kept} more messages not recorded (only the first ${MAX_PROMPT_MESSAGES} are kept)`);
+            }
+            this.#prompt = sizes;
+        }
+        catch {
+            this.#problem("prompt not recorded: reading the messages failed");
+        }
+    }
+    // Problems name the message by number only: its role or text could be anything.
+    #size(message, n) {
+        let role;
+        let content;
+        try {
+            ({ role, content } = (message ?? {}));
+        }
+        catch {
+            this.#problem(`prompt message ${n}: reading it failed; recorded as "?" with no size`);
+            return { role: "?", bytes: null };
+        }
+        if (!isValidRole(role)) {
+            this.#problem(`prompt message ${n}: role isn't a lowercase name of up to 32 letters or _ (e.g. system, user, assistant); recorded as "?"`);
+        }
+        if (typeof content !== "string")
+            this.#problem(`prompt message ${n}: content isn't a string; size not measured`);
+        return { role: isValidRole(role) ? role : "?", bytes: typeof content === "string" ? utf8.encode(content).length : null };
+    }
     // Each problem once; after MAX_PROBLEMS, one line says the rest were dropped.
     #problem(text) {
         if (this.#problems.includes(text) || this.#problems.length > MAX_PROBLEMS)
@@ -81,6 +123,7 @@ export class CaliperRun {
                 status: this.status,
                 marks: { ...this.#marks },
                 ...(this.#problems.length && { problems: [...this.#problems] }),
+                ...(this.#prompt && { prompt: { messages: this.#prompt.map((m) => ({ ...m })) } }),
             };
             saved = Promise.resolve(this.#store.put(this.id, record)).catch(logFailure);
         }

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "caliper-timings",
-  "version": "0.1.0",
+  "version": "0.2.0",
   "description": "Server side of Caliper's server timings: records when a marked request reaches each step, for caliper-ai to read back.",
   "keywords": [
     "caliper",