@vimoxshah/tokenflow 1.1.2 → 1.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (86) hide show
  1. package/CHANGELOG.md +180 -0
  2. package/Dockerfile.team +20 -0
  3. package/README.md +30 -11
  4. package/bin/tokenflow.js +147 -12
  5. package/design/tokens.yaml +330 -0
  6. package/docs/architecture.md +5 -4
  7. package/docs/cli.md +204 -0
  8. package/docs/configuration.md +117 -2
  9. package/docs/design-system.md +187 -0
  10. package/docs/exports-and-budgets.md +85 -0
  11. package/docs/guard-codex.md +132 -0
  12. package/docs/ledger.md +144 -0
  13. package/docs/live-mode.md +40 -0
  14. package/docs/media/overview-aurora-dark.png +0 -0
  15. package/docs/media/receipts-aurora-dark.png +0 -0
  16. package/docs/providers-otel.md +179 -0
  17. package/docs/providers.md +54 -1
  18. package/docs/receipt-schema.md +74 -0
  19. package/docs/roadmap.md +182 -0
  20. package/docs/team-server.md +170 -0
  21. package/docs/ui-views.md +322 -0
  22. package/package.json +7 -2
  23. package/schemas/receipt.v0.json +160 -0
  24. package/scripts/build-menubar-app.sh +3 -1
  25. package/scripts/design-build.js +475 -0
  26. package/src/analytics/anatomy.js +467 -0
  27. package/src/analytics/branch-compare.js +159 -0
  28. package/src/analytics/cache-health.js +141 -0
  29. package/src/analytics/live-view.js +266 -0
  30. package/src/analytics/receipt-schema.js +214 -0
  31. package/src/analytics/receipt.js +709 -0
  32. package/src/analytics/rhythm.js +184 -0
  33. package/src/analytics/whatif.js +263 -0
  34. package/src/commands/budget-scopes.js +133 -0
  35. package/src/commands/doctor-checks.js +400 -0
  36. package/src/commands/guard.js +531 -0
  37. package/src/commands/hooks.js +238 -0
  38. package/src/commands/pricing-diff.js +316 -0
  39. package/src/commands/receipt.js +226 -0
  40. package/src/commands/team-serve.js +407 -0
  41. package/src/commands/week.js +86 -0
  42. package/src/core/annotations.js +97 -0
  43. package/src/core/budget.js +33 -0
  44. package/src/core/bundle.js +45 -2
  45. package/src/core/ingest.js +33 -0
  46. package/src/core/live-status.js +227 -2
  47. package/src/core/policy.js +103 -0
  48. package/src/core/receipt-note.js +123 -0
  49. package/src/core/repo.js +64 -0
  50. package/src/core/sync.js +163 -26
  51. package/src/core/team.js +0 -0
  52. package/src/export/html-snapshot.js +28 -1
  53. package/src/export/menubar.js +21 -0
  54. package/src/export/receipt-card.js +210 -0
  55. package/src/export/week-card.js +185 -0
  56. package/src/providers/mock/index.js +383 -52
  57. package/src/providers/openai/index.js +31 -1
  58. package/src/providers/otel/index.js +656 -0
  59. package/src/server/routes/annotations.js +42 -0
  60. package/src/server/routes/cache-health.js +95 -0
  61. package/src/server/routes/index.js +54 -0
  62. package/src/server/routes/session.js +157 -0
  63. package/src/server/server.js +47 -1
  64. package/src/ui/app.js +541 -308
  65. package/src/ui/charts.js +95 -0
  66. package/src/ui/first-run.js +144 -0
  67. package/src/ui/index.html +4 -1
  68. package/src/ui/palette.js +335 -0
  69. package/src/ui/styles/anatomy.css +117 -0
  70. package/src/ui/styles/annotations.css +40 -0
  71. package/src/ui/styles/branches.css +99 -0
  72. package/src/ui/styles/cache.css +6 -0
  73. package/src/ui/styles/first-run.css +31 -0
  74. package/src/ui/styles/live.css +100 -0
  75. package/src/ui/styles/palette.css +85 -0
  76. package/src/ui/styles/rhythm.css +8 -0
  77. package/src/ui/styles/whatif.css +55 -0
  78. package/src/ui/styles.css +303 -196
  79. package/src/ui/views/anatomy.js +567 -0
  80. package/src/ui/views/annotations.js +121 -0
  81. package/src/ui/views/branches.js +304 -0
  82. package/src/ui/views/cache.js +232 -0
  83. package/src/ui/views/index.js +85 -0
  84. package/src/ui/views/live.js +683 -0
  85. package/src/ui/views/rhythm.js +206 -0
  86. package/src/ui/views/whatif.js +196 -0
@@ -0,0 +1,182 @@
1
+ # Roadmap — TokenFlow after the pivot
2
+
3
+ Written 2026-09-05. This is the ambitious version. Everything here keeps the one
4
+ promise the product is built on: **your data stays on machines you own.** A folder
5
+ you already sync and a server you run on your own network are in. A hosted
6
+ service that holds anyone's usage is out, for now, and the line is drawn in
7
+ section 9.
8
+
9
+ The spine is three products on one engine. Every other item exists to show,
10
+ capture, or distribute one of them.
11
+
12
+ | | Receipts | Guard | Ledger |
13
+ |---|---|---|---|
14
+ | Question it answers | What did this branch / PR cost? | Is the session I am in getting expensive? | What does our team's AI spend buy, per merged PR? |
15
+ | Who reads it | the whole team, in the PR | the developer, mid-session | the engineering manager, the FinOps lead |
16
+ | State today | CLI `tokenflow receipt`, merge-bounded, worktree-aware | CLI `tokenflow guard`, Claude Code hook, incremental | `tokenflow team` over a shared folder or `tokenflow team serve`; receipts joined per branch, the PR join still needs a cached PR list |
17
+
18
+ ---
19
+
20
+ ## 1. Design system — one source, three surfaces (now)
21
+
22
+ Done first because every item below renders through it.
23
+
24
+ - `design/tokens.yaml` is the single source: roles not values, two axes (mode ×
25
+ skin), type scale with size-specific tracking, a 4-pt space scale, radii, two
26
+ motion tiers, status colors, authored chart ramps.
27
+ - `scripts/design-build.js` compiles it into the dashboard CSS, the landing CSS
28
+ and the menu bar's Swift constants, and refuses to emit anything that fails
29
+ the gates: ink contrast, series-on-surface contrast, adjacent-series
30
+ perceptual distance, sequential monotonicity, status-vs-series distinctness.
31
+ - A test regenerates and diffs, so a hand edit to a generated block fails CI.
32
+ - Doctrine lives in `docs/design-system.md`.
33
+
34
+ ## 2. Dashboard — from "a dashboard" to "it told me something" (now → next)
35
+
36
+ **Now**
37
+
38
+ - **Story strip.** The Overview opens with the three sentences that matter this
39
+ week, each with its number, generated from `insights.js` — not a grid of
40
+ KPIs you have to interpret.
41
+ - **Receipts view.** Per repository, per branch: spend, context share, sessions,
42
+ turns, subagent share, vs-median. Click a branch for the PR-comment receipt
43
+ and a copy button. Works in the offline snapshot because receipts ship in
44
+ the bundle.
45
+ - **Polish pass** through the tokens: press feedback on every pressable,
46
+ popovers that scale from their trigger, tooltips that are instant once one is
47
+ open, a short stagger only when a view changes, translucent header, hover
48
+ states gated to pointer devices, reduced-motion that degrades to a complete
49
+ state.
50
+
51
+ **Next**
52
+
53
+ - **Session anatomy.** Pick a session and see its turn-by-turn cost waterfall,
54
+ its context growing turn over turn, its subagent fan-out as a tree, and the
55
+ moment its per-turn cost stepped up. This is the flame graph for agent
56
+ spend; nothing else draws it.
57
+ - **Live sessions.** Transcripts modified in the last few minutes, each with a
58
+ running cost ticker, a context gauge and the guard's verdict. The dashboard
59
+ becomes something you leave open.
60
+ - **Cache health.** Cache-write churn (a system prompt change invalidating the
61
+ cache), 5-minute vs 1-hour write split, hit-rate over time, and the dollar
62
+ cost of each churn event.
63
+ - **Model what-if.** Re-price the same tokens at another model's rates and show
64
+ the difference, labelled as a what-if about price, never a claim about
65
+ quality.
66
+ - **Compare v2.** Any two windows or any two branches side by side, with the
67
+ diverging ramp doing the work.
68
+ - **Rhythm and focus.** Deep-work sessions, project switching per day, the
69
+ hour your marginal cost per turn is highest.
70
+ - **Command palette (⌘K)** for every filter, tab and export; no animation on
71
+ keyboard actions.
72
+ - **First-run screen.** "Here is what we found on this machine, here is what we
73
+ did not, here is why." Sets expectations before the first chart.
74
+ - **Annotations.** Mark a day ("switched to Opus 5", "started using
75
+ subagents") and see it on every time series.
76
+
77
+ ## 3. Menu bar — the glanceable surface (now → next)
78
+
79
+ **Now**
80
+
81
+ - Consume the generated Swift tokens; retire the hardcoded gradients (the
82
+ streak banner's saturated fill breaks the system's own rule that a gradient
83
+ sits only behind a single number).
84
+
85
+ **Next**
86
+
87
+ - **Live session ticker.** The session running right now: spend, context size,
88
+ guard state. Updates every refresh cycle.
89
+ - **Today's receipts.** The three branches that cost most today.
90
+ - **Guard state.** Thresholds, last verdict, one click to raise or clear a cap.
91
+ - **Transient alert.** When a session crosses a declared cap, a small
92
+ notification-style card slides in from the status item and leaves the same
93
+ way it came. Spatially consistent, interruptible, never modal.
94
+ - **Density modes.** Compact for a laptop screen, comfortable for a display.
95
+ - **Keyboard shortcut** to open the popover; no open/close animation on it.
96
+ - **Sparklines** for the last 24 hours of spend, by source.
97
+
98
+ ## 4. Capture — read more, still read nothing sensitive (next)
99
+
100
+ New adapters, each ~80 lines plus a fixture, each reading counts and metadata
101
+ and discarding content:
102
+
103
+ - Gemini CLI, GitHub Copilot CLI, Aider, Antigravity, Zed, Windsurf, JetBrains
104
+ AI Assistant, Continue, Ollama (measured local tokens), LM Studio,
105
+ LiteLLM/OpenRouter local logs.
106
+ - **Codex branch capture** so Codex work can be attributed to a PR (today its
107
+ records carry no branch).
108
+ - Fix the two attribution defects in the adapters: git worktrees counted as
109
+ separate repositories, and `project` set to the working directory's basename.
110
+
111
+ Hooks and integrations that run on the developer's machine:
112
+
113
+ - **Receipt on push.** A `pre-push` hook writes the branch receipt to
114
+ `.tokenflow/receipts/<branch>.json` and adds an `AI-Cost:` commit trailer.
115
+ - **GitHub Action `tokenflow/receipt-comment`.** Reads that file on
116
+ `pull_request` and posts the markdown block. No server involved.
117
+ - **Codex guard.** The same circuit breaker for Codex sessions.
118
+ - **Per-repo policy file** `.tokenflow/policy.yaml`, committed with the code,
119
+ so a team's caps travel with the repository.
120
+ - **Receipt schema v0**, published, so FinOps tools can ingest receipts.
121
+
122
+ ## 5. Ledger — the buyer's view (next → later)
123
+
124
+ - Join receipts into the team rollup: cost per merged PR by team, repo and
125
+ month; the concentration curve for the team; orphan (after-merge) spend.
126
+ - **Self-hosted team server.** One process on a machine the team owns (LAN,
127
+ Docker), fed by the same per-machine rollups the folder sync uses today.
128
+ Per-developer names remain opt-in per person. Still no cloud.
129
+ - **Exports that travel.** A receipt as a PNG card for Slack or a PR; a weekly
130
+ "your AI week" card; CSV of receipts.
131
+ - **Budgets per repo and per team**, using the existing budget engine.
132
+
133
+ ## 6. Landing page and story (now)
134
+
135
+ - Rewrite around the finding, not the features: the cost of every PR, the
136
+ turn-50 step in marginal cost, the guard that acts on it, and the honesty
137
+ caveats that come with each number.
138
+ - Self-hosted fonts, no third-party requests, no decorative 3D background.
139
+ - The live demo shows a Receipts view with synthetic branches.
140
+
141
+ ## 7. Platform reach (later)
142
+
143
+ - **Windows and Linux tray apps.** Same status file, same tokens; native per
144
+ platform rather than a web wrapper, to keep the zero-dependency promise.
145
+ - **Browser extension** that reads token counts from chat UIs' own responses,
146
+ locally, and writes them into the store. Counts only.
147
+ - **Signed macOS build** once there is an Apple Developer account behind the
148
+ project; the Gatekeeper dance is the first thing every new user hits.
149
+
150
+ ## 8. Data quality and trust (continuous)
151
+
152
+ - Price table refresh flow that shows the diff and its sources before applying.
153
+ - Coverage shown on every receipt and every total, as now.
154
+ - Hermes and other session-level sources labelled as such in per-turn views.
155
+ - A `tokenflow doctor` check for every defect found in the 2026-09 audit.
156
+
157
+ ## 9. The line we do not cross yet
158
+
159
+ | In | Out, for now |
160
+ |---|---|
161
+ | Folder sync you already run (iCloud, Dropbox, Syncthing, a git repo) | Any hosted service that stores usage |
162
+ | A server you run on your own network | Telemetry of any kind |
163
+ | Receipts you choose to push to your own repository | Prompt or code content in any store |
164
+ | Exports you choose to share | Default-on names in team views |
165
+
166
+ ---
167
+
168
+ ## Sequencing
169
+
170
+ | Window | Ships |
171
+ |---|---|
172
+ | **Now** (this release) | Design system + compiler + gates (shipped) · dashboard polish (shipped) · story strip (shipped) · Receipts view (shipped) · menu bar on tokens (shipped) · landing v2 (shipped) |
173
+ | **Next** (weeks 2-6) | Session anatomy (shipped) · live sessions (shipped) · receipt-on-push hook + GitHub Action (shipped) · Codex branch capture (shipped) · attribution defect fixes (shipped) · cache health (shipped) · guard state in menu bar (shipped) |
174
+ | **Later** (quarter) | Ledger join + self-hosted team server (shipped) · model what-if (shipped) · new adapters in batches (otel shipped; the rest still need a sample, see docs/providers-otel.md) · Windows/Linux trays (not started) · signed build (not started) |
175
+
176
+ **Status 2026-09-06:** everything in the Now and Next rows above has shipped, along with the
177
+ Ledger, the self-hosted team server, model what-if, and the otel adapter from Later. The six
178
+ views named in section 2's Next list (Session anatomy, Cache health, Model what-if, Compare v2,
179
+ Rhythm and focus, Annotations) all landed as registered modules, plus Compare branches as a
180
+ seventh. Remaining open items: Windows/Linux trays, the browser extension, a signed macOS build,
181
+ and adapters for Aider/Ollama/LM Studio/Zed/Windsurf/JetBrains/Continue/Copilot CLI/Antigravity
182
+ (each needs a sample before it can be verified).
@@ -0,0 +1,170 @@
1
+ # Team server (self-hosted)
2
+
3
+ `tokenflow team serve` runs one process on a machine your team owns — a spare
4
+ box on the LAN, or a container — and lets every teammate's `tokenflow sync`
5
+ push its daily rollup there instead of writing into a shared folder
6
+ (iCloud/Dropbox/Syncthing). It is a second transport for the exact same data
7
+ the folder-sync `tokenflow team` view already reads: the aggregation logic
8
+ (`src/core/team.js`) and the plain-text renderer are unchanged.
9
+
10
+ Nothing here reads prompt or code content. Each machine sends the same
11
+ coarse daily rollup the folder sync already produces — tokens, requests,
12
+ estimated cost, and (only if that developer opted in locally) their chosen
13
+ name. Request bodies are never logged.
14
+
15
+ ## Run it
16
+
17
+ ```
18
+ tokenflow team serve
19
+ ```
20
+
21
+ Defaults: binds `127.0.0.1:7790`, stores uploads under
22
+ `<TOKENFLOW_HOME>/team` (created if missing).
23
+
24
+ ```
25
+ tokenflow team serve --host 0.0.0.0 --port 7790 --token <shared-secret>
26
+ ```
27
+
28
+ A **non-loopback** `--host` always requires a token — the server refuses to
29
+ start without one, so an operator cannot accidentally expose an open write
30
+ endpoint to the network. The token can also come from the
31
+ `TOKENFLOW_TEAM_TOKEN` environment variable instead of `--token` (handy for
32
+ Docker/systemd where flags end up in `ps` output).
33
+
34
+ | flag | default | meaning |
35
+ | --- | --- | --- |
36
+ | `--host` | `127.0.0.1` | bind address |
37
+ | `--port` | `7790` | bind port |
38
+ | `--dir` | `<TOKENFLOW_HOME>/team` | where uploaded rollups are stored |
39
+ | `--token` | — (falls back to `TOKENFLOW_TEAM_TOKEN`) | shared bearer token |
40
+
41
+ ## Auth model
42
+
43
+ There is exactly one shared secret — no per-person accounts, the same token
44
+ for every machine that pushes and every teammate who reads.
45
+
46
+ - **Writes** — `POST /api/rollup` requires `Authorization: Bearer <token>`
47
+ **whenever a token is configured.**
48
+ - **Reads** — `GET /api/team` and `GET /` require the token too, once
49
+ configured: the aggregate carries repo names, branch names, machine names
50
+ and costs, and a LAN peer should not see any of that without the secret.
51
+ Two ways to authenticate a read:
52
+ - `Authorization: Bearer <token>`, same as writes, or
53
+ - a `tf_token` cookie. Visit `/?token=<token>` once from a browser you
54
+ trust; the server mints the cookie (`HttpOnly`, `SameSite=Strict`,
55
+ `Path=/`) and immediately redirects to `/` with the query string
56
+ stripped, so the token never lingers in the address bar or browser
57
+ history. Every subsequent `GET /` from that browser is authenticated
58
+ automatically.
59
+ - Without either, `GET /api/team` returns a 401 JSON error and `GET /`
60
+ returns a minimal "a token is required" page with **no aggregate
61
+ content** — not even a hint of machine names or numbers.
62
+ - **`GET /health`** always answers (so a liveness check never needs the
63
+ token), but the payload depends on auth: `{ "ok": true }` only when
64
+ unauthenticated, the full `{ "ok": true, "machines": <n>, "updatedAt":
65
+ "<iso|null>" }` once authenticated.
66
+ - **No token configured** (the loopback default) — every route stays open,
67
+ unchanged from a server with no auth story at all. There is no per-viewer
68
+ identity to check on a machine only its owner can reach.
69
+ - Put this server on a private network (LAN, VPN, or a Docker network with
70
+ no public port mapping) regardless — the token protects the data, not the
71
+ network path to it.
72
+
73
+ ## Routes
74
+
75
+ - `POST /api/rollup` — body `{ "machineId": "<id>", "files": { "<name>": "<text>" } }`.
76
+ - `machineId` must match `^[A-Za-z0-9_-]{8,64}$` (the same id
77
+ `tokenflow sync` already generates per machine).
78
+ - Every key in `files` must equal exactly `<machineId>.jsonl` or
79
+ `<machineId>.receipts.json` — nothing else is accepted, and nothing is
80
+ ever written outside the server's storage directory.
81
+ - The whole request body is capped at 8 MB.
82
+ - `.jsonl` content is validated line by line (each non-blank line must
83
+ parse as JSON); `.receipts.json` content must parse as JSON as a whole.
84
+ - Files are written atomically (temp file + rename) so a reader never sees
85
+ a partial upload.
86
+ - Response: `{ "ok": true, "files": ["<name>", ...] }`. Invalid requests
87
+ get a short JSON error and a 400/401/413.
88
+ - `GET /api/team` — `aggregate(dir)` as JSON (same shape `tokenflow team
89
+ --json` prints). 401 JSON error when a token is configured and the request
90
+ isn't authenticated (see "Auth model").
91
+ - `GET /health` — counted directly from the uploaded `.jsonl` files (not the
92
+ aggregate), so it reflects what the server has actually received.
93
+ Unauthenticated response is `{ "ok": true }`; authenticated (or no token
94
+ configured) response is `{ "ok": true, "machines": <n>, "updatedAt":
95
+ "<iso|null>" }`.
96
+ - `GET /` — a self-contained HTML page (no external requests, no scripts):
97
+ the dashboard's generated design tokens inlined so colours/fonts match the
98
+ rest of TokenFlow, a heading, machine count and freshness, and the same
99
+ plain-text view `tokenflow team` prints. A "top repositories" table is
100
+ added once the aggregate exposes `receipts.byRepo`; today `aggregate()`
101
+ only reads the `.jsonl` files, so the page shows the `<pre>` view alone
102
+ until the receipts pipeline lands (see "Open questions" below) — the page
103
+ degrades to that automatically, no error either way. When a token is
104
+ configured and the request isn't authenticated, this route instead
105
+ returns a minimal "token required" page with no aggregate content, unless
106
+ the request is `/?token=<token>` (see "Auth model").
107
+
108
+ ## Example: pushing from a machine
109
+
110
+ ```bash
111
+ curl -X POST http://team-box.local:7790/api/rollup \
112
+ -H "authorization: Bearer $TOKENFLOW_TEAM_TOKEN" \
113
+ -H "content-type: application/json" \
114
+ -d '{
115
+ "machineId": "m-a1b2c3d4",
116
+ "files": {
117
+ "m-a1b2c3d4.jsonl": "{\"machineId\":\"m-a1b2c3d4\",\"date\":\"2026-09-05\",\"inputTokens\":1000,\"outputTokens\":200,\"requests\":5,\"estCostUsd\":1.23}\n",
118
+ "m-a1b2c3d4.receipts.json": "{}"
119
+ }
120
+ }'
121
+ ```
122
+
123
+ ## Safety notes
124
+
125
+ - The server never serves files from its storage directory directly — only
126
+ through `aggregate()`/`renderText()`, and never logs request bodies.
127
+ - Only `POST /api/rollup` is rate-capped (by body size, not frequency) —
128
+ nothing else limits request rate. Keep the server on a network only your
129
+ team can reach.
130
+ - `Cache-Control: no-store` is set on every response.
131
+ - The `tf_token` cookie has no `Secure` attribute, since this server is
132
+ typically plain HTTP on a LAN/Docker network rather than behind TLS; if you
133
+ do put it behind HTTPS, note that a plain-HTTP fallback path on the same
134
+ host would still see the cookie (`SameSite=Strict` limits cross-site use,
135
+ not cross-scheme use on the same site).
136
+
137
+ ## Docker
138
+
139
+ `Dockerfile.team` builds a tiny `node:22-alpine` image with no `npm install`
140
+ step (zero runtime dependencies — the whole app is already plain ESM). Set
141
+ `TOKENFLOW_TEAM_TOKEN` and mount a volume for the storage directory:
142
+
143
+ ```bash
144
+ docker build -f Dockerfile.team -t tokenflow-team .
145
+ docker run -d \
146
+ -e TOKENFLOW_TEAM_TOKEN=<shared-secret> \
147
+ -p 7790:7790 \
148
+ -v tokenflow-team-data:/data \
149
+ tokenflow-team
150
+ ```
151
+
152
+ Inside the container the server binds `0.0.0.0` (so Docker's port mapping can
153
+ reach it), which is why `TOKENFLOW_TEAM_TOKEN` is mandatory there.
154
+
155
+ ## Open questions
156
+
157
+ - The exact shape of `receipts.byRepo` on the `aggregate()` result is owned
158
+ by the in-progress receipts pipeline (see `src/core/team.js`,
159
+ `src/core/sync.js` module headers). This page reads `repo`/`name`,
160
+ `tokens`, and `estCostUsd`/`cost` off each entry defensively and skips the
161
+ table entirely if the shape isn't there yet; once that pipeline lands, the
162
+ table may need its column list revisited against the real field names.
163
+ - `--port` with a non-numeric value is passed straight to `server.listen()`,
164
+ which fails with Node's own error rather than a friendly CLI hint.
165
+ - A client that declares a huge `Content-Length` but never times out its
166
+ upload is read to completion (bounded memory, unbounded time) before the
167
+ 413 is returned — see the comment on `readBodyLimited` in
168
+ `src/commands/team-serve.js` for why closing the connection early instead
169
+ resets it. There is deliberately no request-rate limiting; per the task's
170
+ own scope this is body-size-only.
@@ -0,0 +1,322 @@
1
+ # Dashboard views and server routes
2
+
3
+ Add a tab to the dashboard, and an API endpoint to serve it, without editing
4
+ `src/ui/app.js` or `src/server/server.js`.
5
+
6
+ Two registries do this:
7
+
8
+ | Registry | File | Adds |
9
+ | --- | --- | --- |
10
+ | Views | `src/ui/views/index.js` | a tab in the dashboard |
11
+ | Routes | `src/server/routes/index.js` | an endpoint on the local server |
12
+
13
+ `src/ui/views/live.js` is the worked example. Read it next to this page.
14
+
15
+ ## Add a view in four steps
16
+
17
+ 1. Write `src/ui/views/<name>.js` with the exports below.
18
+ 2. Write `src/ui/styles/<name>.css` if the tab needs styles of its own.
19
+ 3. Import the module in `src/ui/views/index.js` and add it to `VIEWS`.
20
+ 4. Run the dashboard and open `#tab=<your id>`.
21
+
22
+ Nothing else changes. app.js merges your tab into the tab bar, links your
23
+ stylesheet in development, inlines it into the offline snapshot, and routes the
24
+ deep link to you.
25
+
26
+ ## The view module
27
+
28
+ ```js
29
+ import { el } from '../charts.js';
30
+
31
+ export const id = 'burn'; // required, unique
32
+ export const label = 'Burn rate'; // required, the tab caption
33
+ export const order = 45; // required, position in the tab bar
34
+ export const css = './styles/burn.css'; // optional, relative to src/ui/
35
+
36
+ export function view(ctx) { // required, returns one element
37
+ return el('div', { class: 'grid' }, [ctx.sectionTitle('Burn rate')]);
38
+ }
39
+
40
+ export function onEnter(ctx) {} // optional, the tab became active
41
+ export function onLeave(ctx) {} // optional, the tab stopped being active
42
+ ```
43
+
44
+ **`id`** must not collide with a built-in tab or another view. app.js drops a
45
+ module with a duplicate or missing id, logs to the console, and keeps every
46
+ other tab working.
47
+
48
+ **`order`** places the tab. The built-in tabs hold these numbers:
49
+
50
+ | Order | Tab | | Order | Tab |
51
+ | --- | --- | --- | --- | --- |
52
+ | 10 | Overview | | 95 | Cache health (registered) |
53
+ | 20 | Receipts | | 100 | Cost |
54
+ | 25 | Session anatomy (registered) | | 105 | What-if (registered) |
55
+ | 30 | Live (registered) | | 110 | Productivity |
56
+ | 40 | Providers | | 115 | Rhythm (registered) |
57
+ | 50 | Models | | 120 | Compare |
58
+ | 60 | Interfaces | | 125 | Compare branches (registered) |
59
+ | 70 | Time patterns | | 130 | Data explorer |
60
+ | 80 | Peaks | | 135 | Annotations (registered) |
61
+ | 90 | Efficiency | | 140 | Data health |
62
+
63
+ They are spaced by 10, so pick a number between two of them. Ties break on id.
64
+
65
+ **`view(ctx)`** returns one `HTMLElement`. It runs on every render, which means
66
+ on every filter change, every theme change, and every tab entry. Keep it a pure
67
+ function of `ctx` and cheap. If it throws, app.js shows a banner in that tab
68
+ only and logs the error, so a broken view costs its own tab and nothing else.
69
+
70
+ **`onEnter` / `onLeave`** fire when the active tab changes, including on the
71
+ first paint and on a deep link. Use them for work that should happen once per
72
+ visit rather than once per render.
73
+
74
+ **Never import `src/ui/app.js`.** Everything you may use arrives in `ctx`. The
75
+ offline snapshot bundler needs an acyclic module graph, and a view that imports
76
+ app.js makes it cyclic. Importing `../charts.js`, `../../core/units.js`, or any
77
+ other leaf module is fine.
78
+
79
+ ## The ViewContext
80
+
81
+ app.js builds a fresh `ctx` for every render. `bundle`, `view` and `filters` are
82
+ getters over live state, so a `ctx` you keep in a closure never goes stale.
83
+
84
+ ### State
85
+
86
+ | Field | What it is |
87
+ | --- | --- |
88
+ | `S` | the whole dashboard state. Read freely. Write only fields you own. |
89
+ | `bundle` | the aggregate bundle the page loaded. |
90
+ | `view` | the computed analytics view for the current filters. This is what most tabs read. |
91
+ | `filters` | the active filter object, the same object as `S.filters`. |
92
+ | `snapshot` | `true` when the page is a saved offline file. |
93
+
94
+ ### Building the page
95
+
96
+ | Helper | Use |
97
+ | --- | --- |
98
+ | `el(tag, attrs, kids)` | build an element. `text:` sets textContent, never HTML. |
99
+ | `card(title, hint, body, actions)` | the standard card. |
100
+ | `chartCard(id, title, hint, renderChart, tableSpec, extraActions)` | a chart with its table twin. Every chart needs one. |
101
+ | `btn(label, onClick, cls, id)` | a button. |
102
+ | `kpi(label, value, sub, opt)` | a KPI tile. |
103
+ | `deltaChip(change, opt)` | the up/down/flat change chip. |
104
+ | `sectionTitle(text)` | a section heading. |
105
+ | `emptyCard(text, detail)` | the empty state. |
106
+ | `openModal(title, body, foot)` / `closeModal()` | the shared dialog. |
107
+ | `drillTo(key, value)` | filter to one value and repaint. |
108
+ | `charts` | everything in `src/ui/charts.js` as a namespace. |
109
+
110
+ ### Formatting
111
+
112
+ `ctx.fmt` holds the formatters from `src/core/units.js`, the same ones the CLI
113
+ uses: `compact`, `int`, `usd`, `pct`, `signedPct`, `shortDate`, `longDate`,
114
+ `hourLabel`, `hourWindow`, `relativeTime`, `humanDuration`, `countdown`, `DOW`.
115
+
116
+ Use them. A number formatted by hand will disagree with `tokenflow status`
117
+ sooner or later.
118
+
119
+ ### Doing things
120
+
121
+ | Helper | Behaviour |
122
+ | --- | --- |
123
+ | `fetchJson(path, opt)` | resolves to the parsed JSON. In a snapshot it resolves to `null` instead of throwing. In the live dashboard it still rejects on an HTTP error, so catch it. |
124
+ | `schedule(fn, ms)` | an interval app.js clears for you. See the lifetimes below. |
125
+ | `rerender(opt)` | recompute the analytics view, then repaint. Pass `{ recompute: false }` to repaint only. |
126
+
127
+ `schedule` has two lifetimes, and which one you get depends on where you call it:
128
+
129
+ * Called from `onEnter`, the interval runs until the tab is left.
130
+ * Called from `view()`, the interval is replaced on the next render. That is
131
+ deliberate: `view()` runs again on every filter change, and a single-lifetime
132
+ timer would stack a duplicate each time.
133
+
134
+ Both are cleared when the tab is left. You never call `clearInterval` yourself.
135
+
136
+ **Register `ctx.schedule` synchronously in `onEnter`, before any `await`.** app.js decides which
137
+ interval belongs to which tab visit at the moment `onEnter` is called, not when it eventually
138
+ finishes, so a call made after an `await` may register against a tab the user has already left.
139
+ `src/ui/views/live.js` is the one registered view that polls, and it calls `ctx.schedule` as the
140
+ very first thing its synchronous (non-`async`) `onEnter` does, before starting its own fetch.
141
+ `cache.js` and `anatomy.js` also keep `onEnter` synchronous, though neither calls `schedule`;
142
+ `annotations.js`'s `onEnter` is `async` but never calls `schedule` either, so the ordering hazard
143
+ does not apply to it. A view that needs both `async` work and a recurring poll should call
144
+ `ctx.schedule` before its first `await`, not after a fetch resolves.
145
+
146
+ ## Working offline
147
+
148
+ Half the value of this dashboard is the saved HTML file. `tokenflow export
149
+ --html` writes one self-contained page that opens from `file://` with no server
150
+ and no network. Your view is inside it, so it must work there.
151
+
152
+ The rule is simple: **anything that needs the server must be optional.**
153
+
154
+ `ctx.snapshot` tells you which world you are in. Follow the Data explorer
155
+ (`viewExplorer` and `loadExplorer` in app.js), which is the reference pattern:
156
+
157
+ ```js
158
+ let rows = [];
159
+
160
+ export async function onEnter(ctx) {
161
+ if (ctx.snapshot) {
162
+ // Read what the exporter embedded. Never call the API.
163
+ rows = (window.__TOKENFLOW_RECORDS__ || []).slice(0, 50);
164
+ } else {
165
+ const res = await ctx.fetchJson('/api/records?limit=50');
166
+ rows = res.rows;
167
+ }
168
+ ctx.rerender({ recompute: false });
169
+ }
170
+ ```
171
+
172
+ Three things fall out of that:
173
+
174
+ * **Branch on `ctx.snapshot` before the call, not in a `catch`.** A snapshot has
175
+ no server. Trying and failing is slow and noisy.
176
+ * **Hide controls that cannot work.** The Live view drops its "Manage limits"
177
+ button in a snapshot, because saving writes to the API. Live also hides the
178
+ whole "Real-time engine" card, because a file has no watcher.
179
+ * **Everything in `ctx.bundle` and `ctx.view` is already there.** The exporter
180
+ embeds the bundle, so any tab computed purely from it works offline with no
181
+ extra effort.
182
+
183
+ If a whole tab is meaningless offline, render `ctx.emptyCard()` with a sentence
184
+ saying why. Do not render a dead control.
185
+
186
+ ## CSS
187
+
188
+ **One file per view, at `src/ui/styles/<view>.css`.** Name it after the view.
189
+ Set `css: './styles/<view>.css'` in the module. Do not add rules to
190
+ `src/ui/styles.css`.
191
+
192
+ * In development app.js injects one `<link>` per view, once, before the data
193
+ loads.
194
+ * In a snapshot `src/export/html-snapshot.js` inlines `styles.css` first, then
195
+ every file in `src/ui/styles/` in name order.
196
+
197
+ Both paths pick your file up automatically. There is no list to update.
198
+
199
+ **Use the token variables only.** They come from the generated block at the top
200
+ of `src/ui/styles.css`, which is built from `design/tokens.yaml`. No raw hex, no
201
+ `rgb()` literals. Colours carry meaning across three skins and two modes, and a
202
+ literal breaks in five of the six combinations.
203
+
204
+ * surfaces: `--surface-1`, `--surface-2`, `--border`, `--hairline`
205
+ * text: `--text-primary`, `--text-muted`
206
+ * data series: `--series-1` to `--series-8`, `--seq-1` to `--seq-7`
207
+ * status only: `--good`, `--warning`, `--critical`
208
+
209
+ Status colours mark status. A chart series never uses one.
210
+
211
+ Keep motion at 300ms or less, and ease out.
212
+
213
+ Prefix your class names with the view id if there is any chance of a clash. All
214
+ view stylesheets load together, in every mode, so `.row` will find someone
215
+ else's `.row`.
216
+
217
+ ## Server routes
218
+
219
+ `src/server/routes/index.js` exports `ROUTES`. Push an entry and the local
220
+ server serves it:
221
+
222
+ ```js
223
+ export const ROUTES = [
224
+ {
225
+ method: 'GET',
226
+ path: '/api/burn',
227
+ handler: (req, res, url, ctx) => ctx.json({ rate: 42 }),
228
+ },
229
+ ];
230
+ ```
231
+
232
+ `path` is matched exactly. `method` is matched case insensitively. There is no
233
+ pattern syntax: use a query string, which `url.searchParams` already parsed for
234
+ you. The handler owns the response from the moment it is called, and nothing
235
+ after it runs.
236
+
237
+ On disk, a route module exports either one `Route` object or an array named
238
+ `<NAME>_ROUTES`, and `src/server/routes/index.js` spreads it into `ROUTES`:
239
+
240
+ ```js
241
+ // src/server/routes/index.js
242
+ import { ANNOTATIONS_ROUTES } from './annotations.js';
243
+ import { SESSION_ROUTE } from './session.js';
244
+
245
+ export const ROUTES = [...ANNOTATIONS_ROUTES, SESSION_ROUTE];
246
+ ```
247
+
248
+ ### A worked example: `GET /api/session`
249
+
250
+ `src/server/routes/session.js` is the reference a request-level route follows. It backs the
251
+ Session anatomy tab, which needs one session's turn-by-turn records — the one thing the bundle
252
+ never carries, because request-level data is too large to ship on every page load.
253
+
254
+ ```js
255
+ // GET /api/session?id=<sessionId>
256
+ export const SESSION_ROUTE = {
257
+ method: 'GET',
258
+ path: '/api/session',
259
+ handler: (req, res, url, ctx) => {
260
+ const id = url.searchParams.get('id');
261
+ if (!id) return ctx.json({ error: 'id is required' }, 400);
262
+ // ...find the session's date range, scan only the months it touches,
263
+ // slim each record to an allow-listed shape, then:
264
+ ctx.json({ id, session, months, scanned, total, returned, cap, truncated, records });
265
+ },
266
+ };
267
+ ```
268
+
269
+ Three things worth copying: the response body is an **allow-list** of fields (`slim()` in
270
+ `session.js` names each one explicitly, so a metadata field added upstream later cannot leak in by
271
+ default); the record count is **capped** (`RECORD_CAP`, 5000) and the cap, the true total and
272
+ whether it was hit all travel in the body rather than silently truncating; and the store scan is
273
+ **narrowed** to only the months the session's own date range touches, never the whole store.
274
+
275
+ ### Handler ctx
276
+
277
+ | Field | What it does |
278
+ | --- | --- |
279
+ | `json(body, code)` | send JSON, default 200. |
280
+ | `send(code, type, body)` | send anything else. |
281
+ | `sendFile(file)` | stream a file from disk. |
282
+ | `readBody()` | resolves to the request body as text. |
283
+ | `config` | the loaded config. Read lazily and cached for this request. |
284
+ | `paths` | the resolved store paths. Same. |
285
+ | `buildBundle()` | the aggregate bundle the UI loads. |
286
+ | `queryRecords(query)` | the record query behind `/api/records`. |
287
+ | `root` | the repository root. |
288
+ | `host`, `port` | what the server actually bound, correct even for port 0. |
289
+
290
+ Throwing from a handler is safe. The server logs it and answers 500.
291
+
292
+ ### Rules
293
+
294
+ * **Do not take a built-in path.** Registered routes are dispatched first, so a
295
+ clash would silently shadow `/api/bundle` or `/api/config` and break the whole
296
+ page. The server prints a warning at startup if you do. Reserved: `/`,
297
+ `/index.html`, everything under `/src/`, and `/api/bundle`, `/api/health`,
298
+ `/api/ping`, `/api/providers`, `/api/records`, `/api/live`, `/api/geo`,
299
+ `/api/config`, `/api/export.csv`, `/api/refresh`, `/api/pricing`,
300
+ `/api/prefs`.
301
+ * **Anything that changes state must be POST.** The server applies its
302
+ same-origin and token check to every non-GET request, before dispatch.
303
+ Registered GET routes get exactly what inline GET routes get, which is
304
+ loopback binding and nothing more. A GET that mutates is unprotected.
305
+ * **Stay local.** No outbound network calls, no telemetry, and never read prompt
306
+ or code content. These hold everywhere in this product and a route is not an
307
+ exception.
308
+ * **A route is never the only way to get the data.** The snapshot has no server.
309
+ If your tab needs a route to say anything at all, it is a tab that does not
310
+ work offline, so make it degrade as described above.
311
+
312
+ ## Checklist before you hand a view over
313
+
314
+ * [ ] `id`, `label`, `order`, `view` exported. `id` unique.
315
+ * [ ] Module added to `VIEWS` in `src/ui/views/index.js`.
316
+ * [ ] No import of `src/ui/app.js`.
317
+ * [ ] Styles in `src/ui/styles/<view>.css`, token variables only.
318
+ * [ ] Every chart has a table twin through `chartCard`.
319
+ * [ ] Opens correctly from `#tab=<id>`.
320
+ * [ ] `tokenflow export --html`, then open the file from `file://`. The tab
321
+ renders, with no console errors.
322
+ * [ ] `npm run lint` and `node --test test/<yours>.test.js` pass.