tokenjam 0.7.0__tar.gz → 0.7.1__tar.gz

{tokenjam-0.7.0 → tokenjam-0.7.1}/.claude/rules/core-architecture.md

@@ -74,7 +74,7 @@ joins the session of any sibling span on the same `trace_id`.
 
   **Rule: both cache fields belong on both sides of every token sum and every price comparison** — an omission that recurs in `core/optimize/`. **(a) In aggregates**, sum cache reads AND cache writes in the same expression; `COALESCE(SUM(input_tokens + output_tokens + cache_tokens + cache_write_tokens), 0)` (the form in `optimize/runner.py`) is canonical for per-cluster and per-session totals — grep `SUM.*cache_tokens` when adding one. **(b) In pricing comparisons**, an actual-vs-alternative-model savings figure must price cache identically on both sides: a hand-rolled alt-price helper omitting cache-write while the actual `cost_usd` bills it inflates every derived saving by the candidate's full cache-write cost, and evades that grep because the SQL is correct and the Python pricing is not — the omission lives in a per-row helper, not a `SUM`. **Keep exactly one pricer:** route every alternative-side computation through `calculate_cost`, which covers all four token classes.
 
-- **`alerts.py`** — `AlertEngine` (the kinds it can fire are the `AlertType` enum in `core/models.py`; read the enum, never a count kept here), `CooldownTracker` (in-memory, per agent+type, resets on restart), `AlertDispatcher` routing to the channel types the `_build_channel` factory knows (stdout, file, ntfy, webhook, Discord, Telegram — that factory is the live set). `AlertEngine.fire()` is the entry point for `SchemaValidator` and `DriftDetector`. Suppressed alerts are still persisted, just not dispatched. Thresholds are module-level constants, not config — read them for values: **retry loop** fires when the same tool call repeats often enough inside a trailing span window (`_RETRY_LOOP_THRESHOLD` / `_RETRY_LOOP_WINDOW`); **failure rate** when the error share over a trailing span window crosses a ceiling, re-evaluated only every Nth error to bound cost (`_FAILURE_RATE_THRESHOLD` / `_FAILURE_RATE_WINDOW` / `_FAILURE_RATE_CHECK_INTERVAL`); **session duration** past a default wall-clock ceiling (`_SESSION_DURATION_DEFAULT`). Stdout and file channels always include full detail regardless of `include_captured_content`.
+- **`alerts.py`** — `AlertEngine` (the kinds it can fire are the `AlertType` enum in `core/models.py`; read the enum, never a count kept here), `CooldownTracker` (in-memory, per agent+type, resets on restart), `AlertDispatcher` routing to the channel types the `_build_channel` factory knows (stdout, file, ntfy, webhook, Discord, Telegram, Mast — that factory is the live set). `AlertEngine.fire()` is the entry point for `SchemaValidator` and `DriftDetector`. Suppressed alerts are still persisted, just not dispatched. Thresholds are module-level constants, not config — read them for values: **retry loop** fires when the same tool call repeats often enough inside a trailing span window (`_RETRY_LOOP_THRESHOLD` / `_RETRY_LOOP_WINDOW`); **failure rate** when the error share over a trailing span window crosses a ceiling, re-evaluated only every Nth error to bound cost (`_FAILURE_RATE_THRESHOLD` / `_FAILURE_RATE_WINDOW` / `_FAILURE_RATE_CHECK_INTERVAL`); **session duration** past a default wall-clock ceiling (`_SESSION_DURATION_DEFAULT`). Stdout and file channels always include full detail regardless of `include_captured_content`.
 
 - **`drift.py`** — `DriftDetector`, Z-score based behavioral drift detection, fires at session end.
 

{tokenjam-0.7.0 → tokenjam-0.7.1}/.claude/rules/web-ui.md

@@ -28,8 +28,11 @@ the sidebar wordmark, and the OpenAPI title, but never in module names, route pa
 Screens: **Overview** (the default landing route — a triage front door), Status, Traces, Cost,
 Alerts, Drift, Optimize (with **Summarize** and **Rules** sub-views), Budget.
 
-`app.py` reads `index.html` into a module string once at `create_app()` time, so editing it requires
-a `tj serve` restart to take effect; tests read the file from disk directly and aren't affected.
+`app.py` caches `index.html` in memory but re-reads it whenever the file's mtime changes (one `stat()` per
+request), so edits show up on the next page reload against a running `tj serve` — no restart needed. The
+`tj-api-key` / `tj-write-token` `<meta>` injection runs on every response, so it applies to the re-read HTML
+too. Changes to `ui/vendor/*` are plain `StaticFiles` and were never cached by the app. Tests read the file
+from disk directly and aren't affected.
 
 - **Offline-first (Critical Rule 18, `.claude/rules/web-ui.md`):** every JS/CSS dep is vendored under `vendor/` — Preact + hooks + htm (ESM via `<script type="importmap">`) and **uPlot** (vendored IIFE global `uPlot` + CSS, pinned in `docs/internal/lens-vendor-versions.md`). No render-time external HTTP. `tests/unit/test_ui_offline.py` enforces this; clickable `<a href>` links are the only allowed external URLs.
 - **Single compute path:** the UI reads everything from the REST API and **never re-implements analysis, aggregation, or plan-tier framing in JS** — it consumes the `framing` block (see `core/framing.py`). If the UI needs a number, extend the endpoint; don't compute it client-side. This applies to gating/filter sets too, not just numbers: if a Python-side map decides which analyzers/findings apply to a persona (e.g. `core/optimize/runner.py`'s `PERSONA_DISABLED_ANALYZERS`), the API already publishes the resolved set (`persona_disabled_analyzers` on `/optimize`) — read that field everywhere the gate is needed, never re-declare the map as a second JS literal. Two call sites deriving the same set from two different sources will silently desync the moment either side edits its copy.

{tokenjam-0.7.0 → tokenjam-0.7.1}/.github/scripts/archive_traffic.py

@@ -14,8 +14,12 @@ automatically by GitHub Actions). The default workflow token works because the
 workflow grants `administration: read`, which the Traffic API requires.
 
 Failure mode: any API error raises and the process exits non-zero, so the run
-shows red in the Actions tab (the Cowork health check then alerts on the missing
-file). No-data weeks still write the file — its existence is the success signal.
+shows red in the Actions tab. **Nothing else watches it** — the health check that
+this docstring used to promise was descoped before the workflow shipped (see
+growth/README.md), and that gap is why a broken publish step went unnoticed from
+2026-W26 to 2026-W41 while this script kept succeeding every week. If you are
+relying on this archive, check the branch rather than assuming a green pipeline.
+No-data weeks still write the file — its existence is the success signal.
 """
 from __future__ import annotations
 

{tokenjam-0.7.0 → tokenjam-0.7.1}/.github/workflows/traffic-archive.yml

@@ -56,15 +56,60 @@ jobs:
           GH_REPO: ${{ github.repository }}
           WEEK: ${{ steps.archive.outputs.iso_week }}
         run: |
+          set -euo pipefail
+          remote="https://x-access-token:${GH_TOKEN}@github.com/${GH_REPO}.git"
           tmp="$(mktemp -d)"
-          git clone --quiet --depth 1 --branch traffic-data \
-            "https://x-access-token:${GH_TOKEN}@github.com/${GH_REPO}.git" "$tmp"
+
+          # Create traffic-data if it is missing rather than dying on it. The
+          # branch disappeared once (deleted, or never recreated after a repo
+          # operation) and every weekly run from 2026-W26 to 2026-W41 then
+          # failed on `fatal: Remote branch traffic-data not found`, AFTER the
+          # collect step had succeeded. Fifteen weeks of snapshots were
+          # gathered and thrown away; the Traffic API retains only 14 days, so
+          # every week before the current window is gone for good. A missing
+          # branch is a recoverable state, so recover from it: the archive's
+          # whole purpose is to outlive that 14-day window, and a publisher
+          # that cannot create its own destination defeats it on the one day
+          # it matters.
+          #
+          # Distinguish "no such branch" from "could not ask". `ls-remote
+          # --exit-code` returns 2 for no matching ref and 128 for a transport
+          # or auth failure, so swallowing every non-zero status would let a
+          # momentary network blip start a fresh local history against a branch
+          # that already exists -- the push then fails as a non-fast-forward,
+          # and the run looks like the very bug this step fixes. Only 2 means
+          # create; anything else re-raises with git's own message.
+          set +e
+          git ls-remote --exit-code --heads "$remote" traffic-data >/dev/null
+          ls_status=$?
+          set -e
+          if [ "$ls_status" -eq 0 ]; then
+            git clone --quiet --depth 1 --branch traffic-data "$remote" "$tmp"
+          elif [ "$ls_status" -ne 2 ]; then
+            echo "error: could not reach ${GH_REPO} to check for traffic-data" \
+                 "(git ls-remote exit ${ls_status}). Refusing to guess that the" \
+                 "branch is missing." >&2
+            git ls-remote --heads "$remote" traffic-data >/dev/null
+            exit "$ls_status"
+          else
+            echo "traffic-data does not exist on ${GH_REPO} — creating it."
+            git -C "$tmp" init --quiet --initial-branch=traffic-data
+            git -C "$tmp" remote add origin "$remote"
+            printf '%s\n' \
+              '# traffic-data' '' \
+              'Data-only orphan branch. Weekly GitHub Traffic API snapshots live under' \
+              '`traffic/<ISO-year>-W<week>.json`, written by .github/workflows/traffic-archive.yml' \
+              'on main. Nothing here is source; do not merge this branch into main.' \
+              > "$tmp/README.md"
+            git -C "$tmp" add README.md
+          fi
+
           mkdir -p "$tmp/traffic"
           cp "traffic/${WEEK}.json" "$tmp/traffic/${WEEK}.json"
           git -C "$tmp" config user.name  "github-actions[bot]"
           git -C "$tmp" config user.email "41898282+github-actions[bot]@users.noreply.github.com"
           git -C "$tmp" add traffic/
-          if git -C "$tmp" diff --cached --quiet; then
+          if git -C "$tmp" diff --cached --quiet && git -C "$tmp" rev-parse HEAD >/dev/null 2>&1; then
             echo "No changes — archive already current for ${WEEK}."
             exit 0
           fi

{tokenjam-0.7.0 → tokenjam-0.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: tokenjam
-Version: 0.7.0
+Version: 0.7.1
 Summary: TokenJam: a local-first, OTel-native cost-saving utility for AI agents
 Project-URL: Homepage, https://tokenjam.dev
 Project-URL: Repository, https://github.com/Metabuilder-Labs/tokenjam
@@ -94,7 +94,7 @@ One command sets up live capture:
 npx tokenjam onboard   # or: pipx install tokenjam && tj onboard
 ```
 
-`tj onboard` asks how you use AI agents and wires that path: live capture, the analyzers your setup can act on, and Lens. Coding-agent users (Claude Code, Codex) also get their recent history backfilled and a zero-token status line wired in; restart the agent and you're live. SDK and API users add `@watch()` to their own code, or point an OTLP exporter at `tj serve`.
+`tj init` is the same command (`tj onboard` is its original name, still there). It asks how you use AI agents and wires that path: live capture, the analyzers your setup can act on, and Lens. Coding-agent users (Claude Code, Codex) also get their recent history backfilled and a zero-token status line wired in; restart the agent and you're live. SDK and API users add `@watch()` to their own code, or point an OTLP exporter at `tj serve`.
 
 Then open the dashboard:
 
@@ -184,9 +184,12 @@ subset with `tj optimize downsize resend relearn`.
 | `verbosity` | — | ✅ | Sessions whose output runs long against a per-(tool, task-shape) baseline |
 | `script` | — | ✅ | Deterministic tool sequences a plain script could replace |
 | `reuse` | — | ✅ | Sessions where your agent re-plans work it has already planned |
+| `shipped` | ✅ | ✅ | What your sessions left behind: commits joined to sessions, what the ones that shipped nothing cost, and what got rewritten |
 
 They find where your agents are overspending. They also tell you where they are not, so you don't spend a week optimizing something that was never costing you anything.
 
+`shipped` is the odd one out, and deliberately so. Every other analyzer asks what a session cost; this one asks what it produced. It joins each session to the commits it made, at a labelled confidence, and reports what the sessions that shipped nothing cost. Nothing in it is a saving: it is measured spend next to measured output, and a session can ship value without a commit. `tj init --hooks` makes commits you type by hand carry a session trailer, which raises how much of your work tj can join. [How the join works →](docs/ledger/overview.md)
+
 That balance is why some checks stay dark for you: when the lever that would recover a category of spend belongs to your harness rather than to you, quoting the figure only makes you feel worse. A check also stays dark when its evidence does not exist on your setup; `summarize` reads agent instruction files off disk, which an SDK or API workload does not have, so it is not offered there. It is also why a quiet result is an answer rather than a failed scan. Optimizing has a price of its own, paid in your attention and sometimes in the agent's output, and a bill lowered by making your agent terser or dumber was never a saving.
 
 ---
@@ -213,6 +216,9 @@ Bench reports measured pass-rate on a suite, never "certified" or "quality prese
 | Full CLI reference, every command and flag | [docs/cli-reference.md](docs/cli-reference.md) |
 | Downsize / Cache / Script / Trim deep-dives | [docs/optimize/](docs/optimize/) |
 | Reuse analyzer deep-dive | [docs/optimize/reuse.md](docs/optimize/reuse.md) |
+| The ledger: sessions joined to commits | [docs/ledger/overview.md](docs/ledger/overview.md) |
+| Commit hooks and git notes (`tj init --hooks` / `--notes`) | [docs/ledger/hooks-and-notes.md](docs/ledger/hooks-and-notes.md) |
+| TokenJam Cloud bridge, and what leaves your machine | [docs/ledger/cloud-bridge.md](docs/ledger/cloud-bridge.md) |
 | Prove a downsize candidate holds (TokenJam Bench) | [tokenjam-bench](https://github.com/Metabuilder-Labs/tokenjam-bench) |
 | Claude Code & Codex integration | [docs/claude-code-integration.md](docs/claude-code-integration.md) |
 | Claude Code vs. Codex vs. SDK vs. OTLP: capability matrix | [docs/agent-capability-matrix.md](docs/agent-capability-matrix.md) |

{tokenjam-0.7.0 → tokenjam-0.7.1}/README.md

@@ -36,7 +36,7 @@ One command sets up live capture:
 npx tokenjam onboard   # or: pipx install tokenjam && tj onboard
 ```
 
-`tj onboard` asks how you use AI agents and wires that path: live capture, the analyzers your setup can act on, and Lens. Coding-agent users (Claude Code, Codex) also get their recent history backfilled and a zero-token status line wired in; restart the agent and you're live. SDK and API users add `@watch()` to their own code, or point an OTLP exporter at `tj serve`.
+`tj init` is the same command (`tj onboard` is its original name, still there). It asks how you use AI agents and wires that path: live capture, the analyzers your setup can act on, and Lens. Coding-agent users (Claude Code, Codex) also get their recent history backfilled and a zero-token status line wired in; restart the agent and you're live. SDK and API users add `@watch()` to their own code, or point an OTLP exporter at `tj serve`.
 
 Then open the dashboard:
 
@@ -126,9 +126,12 @@ subset with `tj optimize downsize resend relearn`.
 | `verbosity` | — | ✅ | Sessions whose output runs long against a per-(tool, task-shape) baseline |
 | `script` | — | ✅ | Deterministic tool sequences a plain script could replace |
 | `reuse` | — | ✅ | Sessions where your agent re-plans work it has already planned |
+| `shipped` | ✅ | ✅ | What your sessions left behind: commits joined to sessions, what the ones that shipped nothing cost, and what got rewritten |
 
 They find where your agents are overspending. They also tell you where they are not, so you don't spend a week optimizing something that was never costing you anything.
 
+`shipped` is the odd one out, and deliberately so. Every other analyzer asks what a session cost; this one asks what it produced. It joins each session to the commits it made, at a labelled confidence, and reports what the sessions that shipped nothing cost. Nothing in it is a saving: it is measured spend next to measured output, and a session can ship value without a commit. `tj init --hooks` makes commits you type by hand carry a session trailer, which raises how much of your work tj can join. [How the join works →](docs/ledger/overview.md)
+
 That balance is why some checks stay dark for you: when the lever that would recover a category of spend belongs to your harness rather than to you, quoting the figure only makes you feel worse. A check also stays dark when its evidence does not exist on your setup; `summarize` reads agent instruction files off disk, which an SDK or API workload does not have, so it is not offered there. It is also why a quiet result is an answer rather than a failed scan. Optimizing has a price of its own, paid in your attention and sometimes in the agent's output, and a bill lowered by making your agent terser or dumber was never a saving.
 
 ---
@@ -155,6 +158,9 @@ Bench reports measured pass-rate on a suite, never "certified" or "quality prese
 | Full CLI reference, every command and flag | [docs/cli-reference.md](docs/cli-reference.md) |
 | Downsize / Cache / Script / Trim deep-dives | [docs/optimize/](docs/optimize/) |
 | Reuse analyzer deep-dive | [docs/optimize/reuse.md](docs/optimize/reuse.md) |
+| The ledger: sessions joined to commits | [docs/ledger/overview.md](docs/ledger/overview.md) |
+| Commit hooks and git notes (`tj init --hooks` / `--notes`) | [docs/ledger/hooks-and-notes.md](docs/ledger/hooks-and-notes.md) |
+| TokenJam Cloud bridge, and what leaves your machine | [docs/ledger/cloud-bridge.md](docs/ledger/cloud-bridge.md) |
 | Prove a downsize candidate holds (TokenJam Bench) | [tokenjam-bench](https://github.com/Metabuilder-Labs/tokenjam-bench) |
 | Claude Code & Codex integration | [docs/claude-code-integration.md](docs/claude-code-integration.md) |
 | Claude Code vs. Codex vs. SDK vs. OTLP: capability matrix | [docs/agent-capability-matrix.md](docs/agent-capability-matrix.md) |

tokenjam-0.7.1/RELEASE_NOTES_0.7.0.md

@@ -0,0 +1,113 @@
+# TokenJam 0.7.0
+
+> Working draft for the GitHub release. Not part of the shipped package.
+
+The largest release since 0.6. TokenJam has always been able to tell you what your agent sessions
+cost. This one starts answering the other half of the question: what those sessions actually
+produced.
+
+```bash
+pipx install --upgrade tokenjam
+tj optimize shipped
+```
+
+## The shipped-value ledger
+
+tj now records which repo a session ran in and who ran it, then joins each session to the commits
+it produced.
+
+- **Repo context and developer identity on every session.** Remote, repo root, branch and HEAD at
+  session start and end, the git author email, and a hashed `developer_id` derived from it. Git
+  shell-outs are read-only, time-bounded, and never on a request path. A value that cannot be
+  resolved is absent rather than guessed, so nothing joins on an invented key.
+- **The session-to-commit join.** A new `session_commits` table, populated on the daemon's
+  background pass. Four match sources: a `git commit` tool call inside the session (`tool_span_git_log`),
+  a `TokenJam-Session:` or `Claude-Session:` trailer (`trailer_session`), a git note naming a
+  session (`git_note`), and an AI co-author trailer inside the session window (`trailer_window`).
+  The first three are `deterministic`; the last is `inferred`. A row is written once at the best
+  confidence found and never downgraded.
+- **The `shipped` analyzer.** `tj optimize shipped` reports how many of the window's sessions
+  shipped a commit to the default branch, the measured cost of the ones that shipped nothing,
+  rework cost (commits whose added lines were mostly deleted again within 14 days), and loop cost
+  (repeated Edit/Write calls on one path with no content change). Sessions also carry a state:
+  shipped, committed, reverted, unshipped, or no repo.
+- **Coverage next to every figure.** The share of your default-branch commits that tj could join.
+  Low coverage means tj is seeing part of your work. It does not mean the rest was unproductive.
+- **Surfaces.** `tj optimize shipped` in the terminal, a shipped line on the `tj status` agent
+  card, `GET /api/v1/shipped`, commits and shipped state on the session detail route, and in Lens
+  the Sessions column and filter, the commit chips with a confidence glyph, the "Shipped this week"
+  tile and the Optimize page's Shipped card.
+
+Nothing in the shipped finding is a saving. Every dollar in it is measured spend on sessions with
+no joined commit, and a session can ship value without a commit: research, review, operations. The
+caveat prints on every surface, and the finding sits outside the recoverable-waste rollup.
+
+## First run and onboarding
+
+- **`tj init` is the primary name.** One command object registered under two names, so `tj init`
+  and `tj onboard` take the same flags, ask the same questions and write the same config. Nothing
+  about `tj onboard` changed.
+- **`tj init --hooks`** installs a `prepare-commit-msg` hook in the current repo so a commit you
+  make from a plain shell mid-session carries a `TokenJam-Session:` trailer. That joins it to the
+  session at deterministic confidence, which is the cheapest way to raise coverage. The hook lives
+  in a managed block, keeps a hook you already have, and refuses a `core.hooksPath` inside the
+  worktree because that is a tracked directory.
+- **`tj init --notes`** adds a `post-commit` hook that writes the session's measured cost to
+  `refs/notes/tokenjam`. It implies `--hooks` and is off unless you ask for it. tj reads
+  `refs/notes/ai` (Git AI) and `refs/notes/exceeds-ink` too, so a repo already annotated by one of
+  those gets a stronger join with no setup. The hook reads the cost through the daemon, so it skips
+  in a few milliseconds when `tj serve` is not running and the commit gets its trailer but no note.
+- **`tj commit-note [sha]`** writes that note by hand, for a commit the hook skipped or for a repo
+  with no hook installed. It never fails the commit it runs from.
+- **`tj init --enforce`** turns on the enforcement proxy in suggest mode and prints what it does
+  and does not touch. Suggest mode forwards every request unmodified and records what a policy
+  would have done. Nothing is blocked or rewritten until you approve it. Subscription-plan traffic
+  is never intercepted and subscription OAuth credentials are never proxied or forwarded.
+- **`tj init --cloud <key> --org <org>`** connects the machine to a TokenJam Cloud organization.
+  The daemon then forwards spans, sessions and the commit joins, resumable from a per-stream
+  high-water mark, so a machine that was offline catches up on its own. Before the first byte
+  leaves, the command prints what does and does not cross and waits for a yes. Token counts, model
+  names, cost, timestamps, tool names, file paths, identifiers, the hashed developer id and the git
+  author email leave. Prompt text, completions, tool outputs, file contents, diffs and secrets do
+  not, unless you turn on both your local capture toggles and `forward_content`.
+  `tj init --cloud off` stops forwarding in place.
+- **Existing installs fill in.** The daemon refills repo context on sessions that were ingested
+  before any of this existed, and `tj init --cloud` refills and matches before its first push. You
+  get history rather than a ledger that starts the day you upgraded.
+- **`tj doctor` and `tj uninstall` cover the new surfaces.** doctor reports hook state per repo,
+  whether a hand commit right now would carry a trailer, and whether the Cloud endpoint is
+  reachable and the key accepted. uninstall strips the managed blocks from every hooks directory tj
+  wrote to.
+
+## Correctness
+
+- An inherited session trailer now ranks below the tool span that ran the commit. A subagent
+  inherits its parent's session id, so a commit the subagent's own Bash tool ran carried the
+  parent's trailer and the parent absorbed it at top confidence. Such a row is now written at
+  `inferred`.
+- Remote normalisation drops the query and fragment, and branch endpoints are picked by timestamp
+  rather than by file order (#762).
+- The Cloud forwarder resumes in arrival order rather than event order, so a backfill that inserts
+  spans older than everything already sent no longer loses them. Resume marks are scoped to the
+  organization and to the database file. A batch the receiver refuses is split until the bad record
+  is isolated, then skipped and counted, so one row cannot wedge the stream.
+- `sessions.updated_at` moves only when a session value actually changes, which stops the bridge
+  re-sending untouched rows forever.
+- The transcript catch-up runs once per process, over a window-bounded tool-span index, with
+  producers keyed by repo.
+- The serve-mode shim gates its write routes with the ingest secret rather than the optional API
+  key, and the backfill hand-off goes through the data-access seam instead of around it.
+- `tj init --cloud` says so when history leaves without an install id.
+- Lens: the hero call to action resolves to the surface that renders the analyzer, instead of a
+  retired route.
+
+## Docs
+
+New pages for the ledger, the commit hooks and notes, and the Cloud bridge, plus the ledger OTel
+attributes in the architecture reference. See `docs/ledger/`.
+
+## Upgrading
+
+`pipx install --upgrade tokenjam`, or `pip install -U tokenjam` in a venv. Schema changes are
+additive, so an existing database is migrated in place. Everything new in this release is opt-in:
+the hooks, the notes and the Cloud bridge stay off until you run the flag that turns them on.

{tokenjam-0.7.0 → tokenjam-0.7.1}/docs/alerts.md

@@ -30,6 +30,12 @@ Configure where alerts go in `.tj/config.toml`. Multiple channels work simultane
 type = "ntfy"
 topic = "my-agent-alerts"
 
+# Mast pager app for iPhone and Android. Free, with or without an account (limits below).
+# The key in the URL is the only credential.
+[[alerts.channels]]
+type = "mast"
+channel_url = "https://mast.tissue.dev/mk_0123456789abcdef0123456789abcdef01234567"
+
 # Discord webhook
 [[alerts.channels]]
 type = "discord"
@@ -56,6 +62,51 @@ path = "~/.tj/alerts.log"
 type = "stdout"
 ```
 
+### Mast
+
+[Mast](https://tissue.systems/mast) is a pager app for iPhone and Android. Install it, make a
+channel, and paste the channel URL into the config above. It works without an account, but a
+channel made that way is lost if the app is reinstalled, and the only sign on this side is a
+"No such channel." line in the log. Sign in to a free Tissue account if you want the channel
+to survive that.
+
+Both ways are free but capped. With no account you get 2 channels and 1,000 messages a month;
+a free account gets 10 channels and 10,000 messages. Mast keeps accepting sends past the monthly
+number, but if your agents alert a lot, check [Mast's limits](https://tissue.systems/mast/docs/limits/)
+before relying on it.
+
+`tj` posts a JSON body with `title`, `body`, `priority` and `key`. Mast documents those fields
+on its [sending page](https://tissue.systems/mast/docs/send/), which is the place to look if
+sends start failing.
+
+`tj` maps severity onto three of Mast's four priorities: `info` sends `quiet`, `warning`
+sends `normal`, `critical` sends `loud`. Nothing maps to the fourth, `page`. It repeats until
+someone acks it and ignores mute and quiet hours, so you have to ask for it, which is what
+you'd do on a channel that's meant to wake someone up:
+
+```toml
+[[alerts.channels]]
+type         = "mast"
+channel_url  = "https://mast.tissue.dev/mk_..."
+priority     = "page"
+min_severity = "critical"
+```
+
+Each channel also has a ceiling, shown as **Highest priority** in the app, and a send that
+asks for more than that is delivered at the ceiling. It's set when the channel is created. The
+iOS app doesn't go below `loud`, the Android app uses whatever default priority you picked, and
+a channel made from an API token starts at `loud`. So a channel created on Android with the
+default of `Normal` will deliver `critical` alerts as `normal`, and `priority = "page"` won't
+page anyone until you raise the ceiling on the phone. Worth checking before you rely on it.
+
+Each send carries a dedupe key built from the same things the cooldown uses: the alert type,
+the agent, and for `cost_budget_session` and `session_duration` the session too. If the same
+alert fires again inside the channel's dedupe window (5 minutes unless you change it), Mast
+adds to the count on the card already on the phone instead of sending a new one. Two sessions
+over their own budget are different alerts, so they get separate cards. The default cooldown
+is 60 seconds, so a repeat usually lands inside the window; if you raise `cooldown_seconds`
+past the window, repeats arrive as new cards.
+
 ## Sensitive actions
 
 Define which tool calls should trigger immediate alerts:
@@ -88,7 +139,7 @@ Cooldown state survives a daemon restart: `AlertEngine` rehydrates it from the p
 
 ## Content stripping
 
-By default, alert payloads sent to external channels (Discord, Telegram, webhook, ntfy) have sensitive content stripped: `prompt_content`, `completion_content`, `tool_input`, `tool_output`. To include full content:
+By default, alert payloads sent to external channels (Discord, Telegram, webhook, ntfy, Mast) have sensitive content stripped: `prompt_content`, `completion_content`, `tool_input`, `tool_output`. To include full content:
 
 ```toml
 [alerts]

{tokenjam-0.7.0 → tokenjam-0.7.1}/docs/architecture.md

@@ -35,7 +35,7 @@ flowchart TD
     Ingest["IngestPipeline\nSanitization · Session continuity · Attribute extraction"]
 
     Ingest --> Cost["CostEngine\npricing.toml"]
-    Ingest --> Alerts["AlertEngine\n13 types · 6 channels"]
+    Ingest --> Alerts["AlertEngine\n13 types · 7 channels"]
     Ingest --> Schema["SchemaValidator\nexplicit JSON Schema"]
 
     Cost --> DB["DuckDB\nlocal · embedded"]
@@ -179,7 +179,7 @@ The alert engine evaluates rules after every span ingest. Alert types fall into
 
 ### Dispatch and cooldown
 
-`AlertDispatcher` routes fired alerts to 6 channel types: stdout, file, ntfy, webhook, Discord, Telegram. `CooldownTracker` prevents alert storms by suppressing repeat alerts of the same type for the same agent within a configurable window. Suppressed alerts are still persisted to the database but not dispatched.
+`AlertDispatcher` routes fired alerts to 7 channel types: stdout, file, ntfy, webhook, Discord, Telegram, Mast. `CooldownTracker` prevents alert storms by suppressing repeat alerts of the same type for the same agent within a configurable window. Suppressed alerts are still persisted to the database but not dispatched.
 
 Content stripping removes `prompt_content`, `completion_content`, `tool_input`, `tool_output` from payloads sent to external channels unless `alerts.include_captured_content = true`. Stdout and file channels always receive the full payload.
 
@@ -426,6 +426,88 @@ Renderers (`tj optimize`, `tj cost`, the web UI cost views) read `pricing_mode`
 
 `SessionRecord.plan_tier` defaults to `unknown` for backfilled rows (no plan signal in the source data). `tj status` surfaces a one-line note when unknown-tier sessions exist; `tj optimize` refuses to render dollar figures for those sessions until the user resolves them via `tj onboard --reconfigure`.
 
+Optimize analyzers do not see unattributed spend; use `tj cost --group-by session` to inspect it.
+
+---
+
+## OTel semconv extensions: repo context and developer identity
+
+The shipped-value ledger needs to know which repo a session ran in and who ran it. That is a second
+group of attribute names, defined in `TjAttributes` and `ResourceAttributes` in
+`tokenjam/otel/semconv.py`, using the standard OTel spelling wherever one exists (`vcs.*`,
+`user.email`, `host.name`) and `tokenjam.*` for the rest. Every one of them is optional: a value
+that cannot be resolved is absent rather than a placeholder, so a downstream join can never match on
+something invented.
+
+**Where the values actually come from.** These names are not emitted by an agent wrapper. There are
+three paths, and it is worth being precise about which is which:
+
+- **Claude Code and Codex.** Nothing is stamped at emission time. The backfill and the daemon's
+  transcript catch-up derive the values from the transcript's `cwd` and branch
+  (`core/repo_context.py::session_context_for_cwd`) and write them straight onto the session row.
+  Sessions ingested before the columns existed are filled in later by
+  `core/transcript_sync.py::refill_session_context`.
+- **Producers that stamp the names themselves.** The ingest path reads them off with
+  `session_context_from_attrs` and stores them on the session. Nothing in the Python or TypeScript
+  SDK sets them for you today, so this is for a producer that chooses to send them, and **the level
+  they have to be set at depends on the route**:
+
+  | Route | Reads from |
+  |---|---|
+  | in-process SDK exporter (`otel/provider.py`) | **resource attributes only** |
+  | Claude Code OTLP logs route (`api/routes/logs.py`) | **resource attributes only** |
+  | OTLP JSON span ingest (`otel/otlp_parsing.py`) | resource **or** span attributes, merged, span wins on conflict |
+
+  Set them at resource level. That is the one placement every route reads. A producer that puts them
+  on individual spans and exports through the in-process SDK loses its repo and developer context
+  silently, because the resource carries none of them and nothing errors.
+- **Outbound to Cloud.** The bridge maps the stored session columns back onto these attribute names
+  on the wire, and adds `tokenjam.install_id` and `host.name` to the resource at that point.
+
+`tj otel-resource-attrs` is **not** part of this. It prints `service.name`, plus
+`service.namespace` when the project is set in config, and nothing else. The per-terminal `claude`
+wrapper exports its output as `OTEL_RESOURCE_ATTRIBUTES`, which is how concurrent terminals land as
+distinct tiles; the ledger columns arrive by the transcript path above, not through that variable.
+
+**The attribute names, per process:**
+
+| Attribute | Value | Source |
+|---|---|---|
+| `user.email` | git author email | `git config user.email` in the session cwd, else the global value, else absent |
+| `tokenjam.developer_id` | `sha256(lower(user.email))[:16]` | derived; the pseudonymous id every ledger surface uses |
+| `tokenjam.install_id` | uuid4, generated once | `~/.tj/install_id`; added to the resource by the Cloud bridge |
+| `host.name` | hostname | `socket.gethostname()`; added to the resource by the Cloud bridge |
+| `tokenjam.github_login` | GitHub login | reserved. Defined in `TjAttributes`, not set by this build |
+| `vcs.repository.url.full` | normalised remote, `https://github.com/org/repo`, no `.git`, no credentials, no query or fragment | `git remote get-url origin` |
+| `vcs.repository.name` | `org/repo` | derived from the remote |
+
+**The attribute names, per session:**
+
+| Attribute | Value |
+|---|---|
+| `vcs.ref.head.name` | branch at session start |
+| `vcs.ref.head.revision` | HEAD sha at session start, only when the source records it. Codex rollouts do; Claude Code transcripts do not, and it is absent rather than guessed. The join keys on repo, branch and time, never on this sha |
+| `tokenjam.repo_root` | absolute path of the repo root |
+| `tokenjam.session.branch_end` | branch at session end |
+| `tokenjam.session.head_end` | HEAD sha at session end |
+
+**Never stamped by the client:** team, department, cost centre, display name. Those are org-side
+maps and have no business on a local span.
+
+`user.email` is the one personal identifier here. It stays on your machine unless you connect
+[the Cloud bridge](ledger/cloud-bridge.md), which states it in the list it prints before the first
+byte leaves.
+
+### Ledger storage
+
+`sessions` carries the same values as nullable columns (`repo_remote`, `repo_root`, `branch_start`,
+`branch_end`, `head_sha_start`, `head_sha_end`, `developer_id`, `user_email`), and the
+`session_commits` table holds the join, keyed `(session_id, commit_sha)` with its `confidence`,
+`source`, author email, commit time and match delta. Three side tables keep every read in SQL: a
+default-branch commit index, per-commit file statistics for the rework figure, and a per-session
+scan watermark. Migrations are additive, as everywhere else in the schema. See
+[the ledger overview](ledger/overview.md).
+
 ---
 
 ## SDK cost-attribution dimensions (multi-tenant cost breakdown)

{tokenjam-0.7.0 → tokenjam-0.7.1}/docs/backfill/langfuse.md

@@ -7,13 +7,15 @@ Imports [Langfuse](https://langfuse.com) observations into the local TokenJam DB
 ```bash
 tj backfill langfuse \
   --source-url https://cloud.langfuse.com \
-  --api-key lf_pk_... \
+  --api-key pk-lf-...:sk-lf-... \
   --since 30d
 ```
 
-Reads from Langfuse's public REST endpoint (`/api/public/observations`) with Bearer-auth. Follows pagination automatically.
+Reads from Langfuse's Observations API v2 (`/api/public/v2/observations`) and follows its cursors automatically. The key is the project's public and secret key joined with a colon (`PUBLIC_KEY:SECRET_KEY`) and is sent with Basic auth. A Bearer public key only gets a scores-only scope in Langfuse, so it cannot read observations; a value without a colon is still sent as a Bearer token for proxies that expect one.
 
-The `--source-url` value can be the bare base URL (`https://cloud.langfuse.com`) — the adapter appends `/api/public/observations`. Self-hosted Langfuse instances work the same way.
+Langfuse Cloud removes the older observations endpoint on 16 November 2026, so v2 is the endpoint to rely on. For a self-hosted server that has no v2 endpoint (it answers 404), the adapter falls back to `/api/public/observations` with page numbers.
+
+The `--source-url` value can be the bare base URL (`https://cloud.langfuse.com`) — the adapter appends the API path. A URL that already ends in `/api/public/observations` still works. Self-hosted Langfuse instances work the same way.
 
 `--since` accepts the same syntax as `tj cost --since`: `30d`, `24h`, or an ISO-8601 timestamp.
 
@@ -45,9 +47,9 @@ Each Langfuse `Observation` becomes one TokenJam span:
 | `type` (`GENERATION` / `SPAN` / `EVENT`) | `kind` + `name` |
 | `startTime` / `endTime` | `start_time` / `end_time` |
 | `model` | `model`, `provider` (derived), `billing_account` (derived) |
-| `usage.input` / `usage.output` | `input_tokens` / `output_tokens` |
+| `usageDetails.input` / `usageDetails.output` (v2), or `usage.input` / `usage.output` (v1) | `input_tokens` / `output_tokens` |
 | `usageDetails.input_cache_read` | `cache_tokens` |
-| `calculatedTotalCost` | `cost_usd` |
+| `totalCost` (v2) or `calculatedTotalCost` (v1) | `cost_usd` |
 | `level` / `statusMessage` | `status_code` / `status_message` |
 
 `billing_account` is derived from the model name (`claude-*` → `anthropic`, `gpt-*`/`o3`/`o4` → `openai`, `gemini-*` → `google`). Unknown models leave it `NULL`; affected sessions will surface as `plan_tier = 'unknown'` in `tj optimize`.