tokenjam 0.6.2__tar.gz → 0.6.3__tar.gz

{tokenjam-0.6.2 → tokenjam-0.6.3}/CLAUDE.md

@@ -203,6 +203,34 @@ Post-ingest hooks run after each span is written to DB:
 
   **A headline figure is the CEILING, not the expectation — and a ceiling still has to be reachable.** Where the arithmetic legitimately supports a range, the number a card leads with is the top of it: the largest figure the derivation honestly bears, not a discounted guess at what a user would actually realize. This is a founder decision and it is deliberate, so do not "correct" a headline downward by applying a realization factor, an adoption rate, or any other haircut invented at the render layer. It is not, however, a licence to publish an unreachable number: a ceiling has to be attainable in *some* real world, so a slice the fix **structurally cannot touch** — spans the remedy does not reach, sessions the analyzer never examined — does not belong inside it, and shaving those out is a correctness fix rather than a haircut. One carve-out: where a card offers a fix the user is about to apply, lead with the realized figure instead, because at that point the honest question has changed from "how large could this be" to "what will this specific action do". This is a convention about which of two defensible numbers to lead with; the numbers themselves belong on the cards and in `estimate_basis`, never in this file.
 
+  **WHERE a permanent rule lands is an INPUT to whether it is offered, not a presentation of the decision.** Four analyzers (`downsize`, `resend`, `subagent`, `relearn`) share one fix shape — a block appended to a `CLAUDE.md` — and each used to offer exactly one destination, the workspace/root file. That is wrong in both directions: a root-level rule is re-sent in every session of every project, including the ones that never exhibited the problem, and it dilutes the file whose dilution is what makes rules start being ignored. Every session records its working directory (`core/transcript.first_recorded_cwd` is the one extractor; `core/summarize/repo_roots.resolve_roots` turns those into boundary-safe roots), so `core/optimize/rule_placement` can resolve which projects incurred a finding and attribute each session's share to a destination. Because the standing cost is `per-session tokens x the sessions that load the file`, confining a rule to the projects that exhibited the behaviour can flip a net-negative verdict to net-positive with no change to the finding at all. Three things fall out and are easy to get wrong. **(a) Two costs, not one.** STANDING cost follows the sessions (what the rule costs to keep, and the only figure the saving is netted against); FOOTPRINT cost follows the FILES (`per-session x destinations written`, and the only figure the write budget is spent in). A three-project split can halve the standing cost while tripling the budget spend; charging either for the other's question makes the placement decision stop being arithmetic. **(b) The growth allowance is per FILE.** `measured_agent_file_tokens` aggregates the `summarize` scan, which is right for "may this window grow the files at all" and wrong for "may this rule grow THIS file" — 10% of an aggregate dominated by a large root file can exceed a small project `CLAUDE.md` entirely, so `measured_agent_file_tokens_by_path` + `WriteBudget.destination_budget` size each destination against its own measured size. **(c) Unifying the FIX surface must not merge the CLAIMS** (Critical Rule 27): the per-session weight maps (`DowngradeFinding.driver_session_tokens`, `ResendFinding.session_weights`) are BREAKDOWNS of each analyzer's own `past_overspend_tokens`, split by the same weights on both the token and the dollar side so every destination's implied rate equals the finding's own (Critical Rule 28). A recorded cwd that no longer exists is counted and surfaced in a `coverage_note` saying the gap is what was **not analysed** (Critical Rule 30), never reconstructed and never redistributed onto the destinations that did resolve.
+
+  **A rule a card writes must ERASE the number the card found.** `past_overspend_usd` is the maximum its analyzer knows: applying the fix should remove the behaviour that produced it. If applying it would leave the number roughly where it was, that is a correctness bug, not a UX gap. The live example: `SUBAGENT_RUBRIC_INTRO` used to end with "a subagent that does little tool work and returns a short result rarely needs the premium tier" — exactly the `output_tokens`/`tool_calls` gate deleted from `analyzers/subagent_rightsizing.py` for inverting on the expensive dispatches (Critical Rule 29), so the rule told the agent to keep doing the thing the card was billing it for. `context_resend.RIGHTSIZE_FIX_TEMPLATE` carried the same sentence in a different costume. Two durable halves: **the shape-based default-down is the lever and stays**, and **a premium escape hatch must be checkable by an outside reader rather than self-assessed** — "unless the subtask genuinely needs deep reasoning" asks the dispatching agent to rate its own task's difficulty, and an agent asked that question answers yes, so the exception swallows the rule. State the exception as conditions that are either written into the dispatch or not.
+
+  **HOW a rule reaches the agent is a DELIVERY MECHANISM, and "rung 3 is free" is FALSE for a context-injecting one.** Appending markdown to a `CLAUDE.md` is one way to put guidance in front of an agent; a Claude Code hook is another, and for several analyzers a better one, because it delivers at the moment of the decision rather than at the top of a long context. `core/rulewrite/delivery.py` is the seam: a `DeliveryKind` owns its own renderer AND its own pricer, and adding a mechanism is a registration plus two functions — never an edit to the staging, diff, apply, undo or budget machinery, none of which may name a CLAUDE.md. The pricing half is the part that bites. `write_budget`'s rung ladder charges rung 3+ ZERO because a settings file is *executed*, never sent as prompt text; that is earned by a formatter or a guard hook, and it is **wrong for a `UserPromptSubmit` re-injection or a `PreToolUse` additionalContext nudge**, which are prompt text on a different SCHEDULE. Their cost is worse-behaved than a rule's, not better: an injected block lands in the conversation and is re-sent every subsequent turn, so it is not even bounded by the number of injections the way a session-start read is bounded by the session count. The ladder grades intervention STRENGTH, not delivery, so the question "does this put tokens in front of the model" is asked of the mechanism (`DeliveryKind.carries_prompt_text`) and never inferred from the rung. Caught in this repo's own first draft of the seam: a rung-3 kind flagged as carrying prompt text still priced to zero, because its pricer routed through the ladder. An unknown mechanism is REFUSED rather than falling back to markdown — a wrong write to a real file is the one failure the lifecycle exists to prevent.
+
+  **A fix the user already dealt with must lose its OFFER and keep its FIGURE.** The rule-write surface consults the apply ledgers (`cost_applied.json` / `applied_fixes.json`, via `cost_apply.applied_signatures` / `relearn_apply.applied_signatures`, matched by `cost_apply.signature_is_applied`) and withdraws the offer for a signature already applied. `past_overspend_usd`/`_tokens` are untouched, and that is Critical Rule 32 in its sharpest form: the waste genuinely happened inside the analyzed window, and the user having since fixed it does not un-spend the money. An action-availability gate establishes that WE have no remedy left to offer, never that the spend was avoidable or free — so it may never reach back and edit a past figure, and a zero there would be the exact "no action for this" / "this was free" conflation. Pin it by asserting the whole record is byte-identical with and without the ledger entry EXCEPT the offer fields; an equal-value check passes a figure that got re-derived through a zero. Two further rules that cost real correctness here: **the row stays listed** carrying an applied state rather than vanishing (a user who applies something and then sees nothing cannot tell "done" from "broken"), and **an unreadable ledger reads as "nothing applied"** — that direction can only waste attention, while the opposite hides fixes the user never made. Reuse `signature_is_applied` rather than writing a second matcher: it already resolves a legacy agent-only mark as covering the later model-qualified signatures for that agent, and a second matcher silently reopens every card it settles.
+
+  **A fix must name an action that EXISTS, for THIS user, through a key the harness actually reads.** Four shipped defects, one framing failure — applying the fix has to be able to erase the number its analyzer found. (a) The right-sizing snippet emitted `reasoning_effort:`, which Claude Code does not read; the field is `effort` (`low`/`medium`/`high`/`xhigh`/`max`). **A silently-ignored key is worse than no line**: it converts an unfixed problem into one the user believes is fixed. It was also hardcoded to `low` for a population that is by construction large and tool-heavy, so the guess recommended making the expensive work worse — a value that cannot be derived from the observation is OMITTED, never guessed, because a plausible line in a frontmatter block is indistinguishable from a measured one. (b) The subagent lane required an agent definition file to already EXIST, and every Claude Code dispatch is a built-in (`general-purpose`, `Explore`, `Plan`, `fork`) which has none — so the branch was unreachable on the dominant corpus and the claim fell through to prose. **A user or project subagent of the same name overrides the built-in and keeps its own `model`, so no-file is not no-fix: offer the CREATE.** The origin belongs in the evidence too — `model` defaults to `inherit`, so an Opus-driven session hands every dispatch an Opus worker unless pinned; that is an unset default, not a decision, which is why it goes unnoticed. Watch the shape checks around this: a lowercase-only slug predicate silently rejected `Explore` and `Plan`, the exact names the fix targets. (c) The per-agent downsize lane told Claude Code users to "redeploy or restart the agent", but there `agent_id` is `claude-code-<cwd-basename>` — a PROJECT DIRECTORY with ephemeral sessions, no process and no model id written down. Gate that with a **persona check at the point of production** (Critical Rule 26(c)), not a dollar threshold, and withhold only the offer. (d) `relearn`'s `edit_before_read` shipped a card whose own fix text says the harness already handles it and no rule is needed. `is_placeholder_fix` cannot see this — the text is a real, specific sentence that happens to say there is nothing to do — so the family carries an explicit `advisory_only` flag and `REASON_ADVISORY_ONLY`, distinct from the placeholder reason because "no template matched" and "none is needed" are different statements. In all four the observation stands and `past_overspend_*` is untouched (Critical Rule 32).
+
+  **Fix text lives in ONE catalog with a lint, because scattered constants are why the above could happen.** `core/fixes/` holds one `FixRecord` per fix type — text, delivery kind, personas, lever, the observation it answers, and the shapes it may not excuse — and every analyzer reads its text from there instead of defining a constant. The proof this is structural, not tidiness: the identical sizing contradiction shipped in BOTH the subagent rubric and the resend right-sizing template in different words, so correcting the reported one left the other live. Two copies of a policy is two policies. `core/fixes/lint.py` checks the properties mechanically — under the published line ceiling, long enough to be an instruction, no escape hatch the agent grades itself against, advisory-only text never offered as a write, fixed-point behaviour escalated to a hook — and above all: **a fix may not re-license the behaviour its own analyzer bills for**, which is the whole D-class and is now caught by a substring check against the record's own `must_not_relicense` set rather than by review. Two dimensions the catalog carries that a bare string cannot: **persona** (a Claude Code user edits instruction files, agent files and hooks; an SDK service edits routing code and redeploys — handing either the other's fix names an action they cannot take) and **lever**, where `model` and `effort` are separate and not substitutes: a worker that did not know enough needs a bigger model, one that did not try hard enough needs more effort.
+
+  **One instruction, one record — and the duplicate check must use CONTAINMENT, not Jaccard.** Three analyzers (`resend`, `downsize`, `relearn`'s `context_overflow`) each authored their own wording of "delegate context-heavy work to a subagent", so three near-identical blocks could land in one `CLAUDE.md`. The write budget's one-block-per-family rule cannot see this: they are three families from three analyzers. The harm is not untidiness — **length and redundancy reduce adherence**, so writing a rule three times makes it LESS likely to be followed than writing it once, and each analyzer's copy actively defeats the others. Their CLAIMS stay disjoint (Critical Rule 27 is untouched — they price different span populations); only the instruction is shared, with per-analyzer framing living in the card's advise text rather than in a second copy of the rule. The lint's metric is load-bearing and the obvious choice is wrong: **Jaccard divides by the union, so it collapses when two texts differ in LENGTH, which is exactly this defect's shape** — measured against the real wordings, the two duplicates scored 24% and 17% Jaccard against the rule they duplicated and would have been certified clean. Containment (`|a ∩ b| / min(|a|,|b|)`) scores the same pairs at 67% and 72%. A check that cannot catch what it was written for is worse than no check, because it reports the problem as absent.
+
+  **Two surfaces rendering one finding must lead with the SAME fix, through the same helper.** The Review inbox card led with the durable offload rule while `tj optimize`'s CLI branch led with `/compact`, for the identical finding — and the CLI led with the weaker one, which `COMPACTION_FIX`'s own text disclaims ("never fixes the pattern going forward; treat it as immediate relief, not the durable fix"). Both now render through `cost_proposals.compound_offload_fix`, so they cannot drift into different wordings again. Related: **`/compact` is never listed as an actionable lever.** It is transient, persists nothing past the session, and users who feel a session filling already do it unprompted — it sits below the actionable ceiling recorded in `.claude/product-state/ANALYZER-PERSONA-MATRIX.md`. Listing it beside durable levers implies it is the same kind of thing, and pads a list whose length is itself a cost to adherence. It survives only in an explicitly-labelled "immediate relief in an already-full session" position.
+
+  **A path-scoped rule is not a cheaper way to say the same thing — it changes WHICH fixes are offerable at all.** A `.claude/rules/*.md` carrying a `paths:` glob loads only when Claude reads a matching file, so only its frontmatter is resident (measured on a real record: ~13% of the same words in a `CLAUDE.md`). Because standing cost is an INPUT to the offer decision, that near-zero rent flips fixes from net-negative to net-positive: the same observation that cannot carry a `CLAUDE.md` rule can carry a scoped one. Three things are load-bearing. **(a) The frontmatter is the entire reason it is cheap** — a rule in the same directory WITHOUT `paths:` loads at launch with `CLAUDE.md` priority, so `load_semantics.classify` takes the file TEXT and decides on `has_paths_scope`, never on the directory; a path-scoped kind that emits no globs has silently become an always-resident rule while still being priced as if it were not, and the renderer refuses rather than degrading because the failure is invisible (the file is written, the rule works, only the cost is wrong). Accept both YAML forms — a block list's inline value is empty, so an inline-only check reads a perfectly good scoped rule as unscoped. **(b) Selection is DERIVED from the LEVER, never preferred.** A rule about the shape of the next action (model, effort, offload, routing) decides something *before any file is read*; scoped to globs it would load only after the decision it governs was made — present, well-formed and useless, which is worse than paying the rent and is invisible. Only file-shaped (awareness) fixes may scope, and only when the record names its own globs; inferring a glob from prose is guessing which files an instruction is about. **(c) Brace expansion has a budget** — past it a pattern is used UNEXPANDED and matches nothing, so the rule reads as applied while never firing; refuse such a glob rather than emitting a silent no-op.
+
+  **Hooks are already written; what was wrong was their PRICE.** `relearn_apply` has shipped hook generation for some time (guard + reactive renderers, the settings patch, the enforcement wiring), so "nothing writes a hook" is not the gap — three of the four rung-3 families that ship are `PostToolUseFailure` reactive hooks that inject `hookSpecificOutput.additionalContext`, and the rung ladder priced all of them at ZERO. For an executing guard that zero is earned; for an injecting hook it is false, and its cost is worse-behaved than a rule's, because the injected block lands in the conversation and is re-sent on every subsequent turn rather than being bounded by session count. Hence two delivery kinds, not one, each with its own pricer. Three rails the generated hook must carry, and they are not interchangeable with a rule's: **`additionalContext` MUST be nested inside `hookSpecificOutput`** (at top level it is silently ignored — the hook runs, exits 0, injects nothing, and looks like it worked); **the per-session cap lives in the script**, keyed on the stdin `session_id`, because only `SessionStart` is structurally once-per-session and the `once` field is honored solely in skill frontmatter, never in a settings file; and **injection is advisory, fail-open, never blocking by default** — a misfiring blocking hook degrades every session until removed, an asymmetry that justifies the rails on its own. Hook entries MERGE across settings layers rather than overriding, so a write cannot clobber the user's hooks but also must not assume it is the only writer.
+
+  **Ground a behavioural rule in the evidence it came from — by REPLACING vague words, never by appending them.** The behavioural fixes (`downsize`'s driver-role card, `resend`'s offload rule, `relearn`'s rung-1 notes) described a HABIT generically while the analyzer that produced them knew which directories the user worked in, which tools drove the tail and which models ran it. That is the shape of instruction the published guidance says gets ignored ("concrete enough to verify": `API handlers live in src/api/handlers/` beats `keep files organized`). Four rules. **(a) Substitution, never generation** — a template carrying "you did X in Y" beats both a generic sentence and a model call: no cost, no latency, no new failure mode, and it cannot hallucinate a directory the user does not have. **(b) SPECIFIC MUST NOT MEAN LONGER.** The guidance is specific *and concise*, and a longer instruction file reduces adherence — the very thing grounding is for — so a substitution REPLACES a vague span and a grounded rendering that came out materially longer is DISCARDED for the generic text (`grounding.MAX_GROUNDED_GROWTH`). On the reference corpus the grounded offload rule is 94% the length of the generic one it replaces, naming three real repos in place of a parenthesised enumeration; the failure mode to guard against is the generic sentence plus an evidence paragraph, which looks like an improvement and is a regression. **(c) Grounding lives in RENDERING, never in the record set** — one catalog record per fix, always; minting a record per user or per finding makes the catalog and its duplicate lint meaningless overnight, so the record declares which spans may be replaced and the renderer fills them. **(d) Degrade per-substitution, and never invent specificity** — an unobserved slot skips its own substitution and keeps that generic span, so a rule can be grounded in its directory even when the tool mix was not observed; an empty field means not observed and is never filled with a plausible-looking value. Mechanical fixes (`deadweight`'s "remove server X", `subagent`'s "set `model:` here") are already concrete by construction and are deliberately left alone — this is specific to rules that describe a habit rather than an edit.
+
+  **A user's DISMISSAL is a decision and belongs server-side — durable, reversible, and figure-safe.** It used to live in the browser (`localStorage`), which means it was not really recorded: a cleared profile, a second browser or a different machine and every dismissed card came back. It now lands in `dismissed.json` beside the DB, the same lock-free sink the applied/reverted ledgers use — and **not** a DuckDB table, for a structural reason rather than a historical one: `tj rules` is in `no_db_commands` and must answer while `tj serve` holds the write lock, and DuckDB permits one writer OR many readers with no concurrent read-only escape, so a dismissal stored in DuckDB could not be READ by the command that has to honour it whenever the daemon is up. Three properties, each a test. **(a) It suppresses the OFFER only** (Critical Rule 32) — the behaviour happened and cost what it cost, and "not this one" is a statement about our recommendation, not about the user's bill; the record is byte-identical with and without the dismissal except the offer fields. **(b) It must be REVERSIBLE, and durability is exactly what makes that non-negotiable** — a dismissal that cannot be undone trades a card that came back on every browser for one that never comes back anywhere, so the row stays listed carrying its state and `undismiss` restores it. **(c) It resolves through the ONE matcher** (`cost_apply.signature_is_applied`) and states its exclusion filter once, like its two sibling ledgers — a dismissal that did not honour the legacy-signature equivalence would let a card dismissed under one signature reappear under its refinement. Browser-local dismissals are not migrated: `localStorage` is unreadable server-side, so they simply lapse, and nothing in the UI pretends otherwise.
+
+  **A flag set UPSTREAM of a pass that rewrites the same field is not a flag, it is a suggestion.** `relearn`'s advisory families set `write_offered=False` at proposal construction, and `_apply_write_budget` then assigned `write_offered=decision.offered` from an allocator that knows nothing about advisory — so the withdrawal held in the unit test (which exercised the constructor) and was silently undone on the live daemon. **Unit tests cannot see this class**: they call the stage that sets the field, not the stage that overwrites it. The fix is to keep such a record out of the later pass entirely rather than to set the field twice. Two general lessons. (a) When a value is written at two stages, the LATER one wins and the earlier one is decoration — either remove the earlier write or exclude the record from the later stage. (b) **A defect that survives a green suite is usually a stage-boundary defect**, so verify through the composed path (here `_apply_write_budget` directly) rather than through the constructor, and boot the daemon against a real corpus before believing a behavioural fix shipped. The same live pass found placement's per-destination session counts computed and never serialized — the payload carried only paths, so every consumer reported `sessions: 0` for destinations the resolver had counted correctly, which is the one figure that justifies placing a rule at all.
+
+  **A gate the user cannot pass is not a gate, it is a dead end — and a ceiling over an AGGREGATE stops being meaningful once the thing it gates can be placed elsewhere.** The write budget's standing ceiling withheld every rule whenever the user's agent files were collectively too large, and told them to compress. Measured on seven real instruction files across six repos, compression returns a ~0.952 prose ratio (much of it whitespace reflow), so the two largest cards this product produces were gated behind a remedy that returns almost nothing. Two fixes, and both generalise. **Name a remedy that works**: these files are long by ACCUMULATION (numbered incident lists, append-only logs), where what frees space is pruning what no longer applies, expiring what was time-bound, and moving file-specific guidance into a path-scoped rule — compression works on prose, and these files are not long because their prose is verbose. **Scope the gate to what it is actually about**: the ceiling now asks whether THIS destination is too large, not whether the corpus is, because one pathological root file was blocking writes into nearly-empty project files with ample headroom. An aggregate gate was correct when there was one destination; placement made it answer a question nobody asked.
+
   **Net-of-standing-cost is part of the contract, not an extra.** A fix that WRITES a permanent artifact into the user's agent files is not free: a rung-1 `CLAUDE.md` rule is re-sent on every future session, forever. Any analyzer whose fix is such a write MUST report `past_overspend_*` **net** of that standing cost and park the pre-net figure on a `gross_*` field; a proposal may never claim a saving larger than its net value. The arithmetic lives in exactly one place, `core/optimize/write_budget.py`, which also owns the write budget (ranked, capped, one block per family, no generic-placeholder rule) and reads the agent-file footprint off the `summarize` analyzer so the product can never recommend adding rules to a file the same report says to compress. Two rules that fall out of it and are easy to get wrong: **rung matters** (rung 2 charges only the always-loaded skill frontmatter, rung 3+ hooks/config charge ZERO because they are executed, never sent as prompt text), and **the budget is spent in PER-SESSION tokens** while the saving is netted against the WINDOW total — charging the window total against the budget would shrink it as a user's session count grows, which is backwards. The 30-day session pace both sides are expressed on is defined once in `core/optimize/projection.py`; never re-derive a projection factor locally.
 
   **The same load semantics govern READING a file, not just writing one.** "Rung matters" is a statement about how Claude Code delivers a file, so it holds identically when an analyzer PRICES an existing file. `core/summarize/load_semantics.py` is the single source of truth both sides classify through: a `CLAUDE.md` / `.claude/rules/*.md` is resident in full on every call of every loading session, while a `.claude/skills/*/SKILL.md`, `.claude/commands/*.md` or `.claude/agents/*.md` surfaces only its **frontmatter** that way and delivers its **body** on invocation. So the window figure is two separately-observed terms — `always-resident reduction x sessions x reads per session` **+** `on-demand reduction x OBSERVED invocations` — never one term applied to the whole file. Getting this wrong in either direction is a real defect: pricing a whole skill body as always-resident made an uninvoked 160 KB skill library the single most expensive prompt file a user owns and pushed the cross-analyzer token rollup to 115% of the window's billed tokens; pricing an invoked body at zero would deny that a long skill that really runs costs anything. The invocation multiplier is OBSERVED from Claude Code transcripts (`core/summarize/invocations.py`: `Skill` tool calls, typed `<command-name>` slash commands, `Task`/`Agent` `subagent_type`), never assumed — and "invoked zero times" (a measurement, prices the body at zero) is a different statement from "no transcript corpus" (not a measurement; Critical Rule 28 corollary (a) degrades BOTH fields).
@@ -228,6 +256,7 @@ Post-ingest hooks run after each span is written to DB:
 - **`tokenjam/api/deps.py`**: `require_api_key` — FastAPI dependency for optional API key auth on GET endpoints. Only enforced when `api.auth.enabled = true` in config.
 - **`tokenjam/api/routes/`**: One file per resource — `spans.py` (OTLP JSON ingest), `traces.py`, `cost.py`, `cost_compare.py`, `tools.py`, `alerts.py`, `drift.py`, `optimize.py`, `reuse.py` (`GET /api/v1/reuse/clusters` — the Reuse finding plus skeleton-rendering extras `planning_texts` + `pricing_mode`; a dedicated endpoint (not bolted onto `/optimize`) so the per-cluster planner text, which can be many KB, isn't paid for on every Overview poll — #154), `budget.py`, `status.py`, `agents.py`, `metrics.py` (Prometheus text format from DB queries), `version.py` (unauthenticated `GET /health` → `{"status":"ok","version":...}` mounted with no prefix, plus `GET /api/v1/version`; the version is derived at runtime via `importlib.metadata.version("tokenjam")` — no hardcoded literal). **The dollar-bearing read routes (`/cost`, `/cost/compare`, `/optimize`, `/budget`) each return a `framing` block** (see `core/framing.py`) so the web UI renders plan-tier-aware figures without re-deriving the rules in JS. `/optimize` takes `?fast=true` to skip the expensive Trim analyzer (returns `skipped_analyzers`) for the polling Overview; `/cost` returns a window-bucketed `series` for the chart (see Web UI below). **Concurrency:** the sync (`def`) read routes (`/optimize`, `/cost/compare`) run in Starlette's threadpool, so concurrent requests reach the DB from multiple threads. `DuckDBBackend.conn` is a **per-thread DuckDB cursor** (`threading.local`) over one shared database — cursors are independent connections safe for concurrent use, so fan-out callers (the Overview) can fetch in parallel. Fixed in #124 (was a single shared connection that aborted under concurrent access); do not collapse `conn` back to one shared connection object.
 - **`tokenjam/core/summarize/`**: Structure-aware prompt summarization (advisory; new feature). `detect.py` classifies prose vs. structure (fenced/inline code, tags, templates, tables); `candidates.py` (+ `catalog.py`/`estimate.py`) powers the `tj summarize list` scan; `wrap.py` is the pure protect→restore algorithm (wrap each structured span behind an id'd `<tj-keep>` marker, restore verbatim by id — structure is a hard guarantee); `session.py` is the no-scratch `prepare`/`check` lifecycle + staging (re-derives the wrap from the live file + a content hash; persists nothing but the staged result); `apply.py` writes a staged rewrite back to the file (default dry-run; `--go` writes) behind an owner + content-hash + symlink guard, with a gzip backup and `undo`; `backup.py` stores the gzipped original + metadata under `~/.tj/summary/backups/`; `delivery.py` is the CLI's automated rewrite step — `claude -p` (subprocess, timeout-guarded) or the Anthropic API (lazy `httpx` + the user's own `TJ_ANTHROPIC_API_KEY`) — plus the "pays for itself" amortization. Pure domain logic — no `tokenjam.cli`/`tokenjam.api` imports (delivery's API path lazily imports `httpx`, its lone outbound dependency).
+- **`tokenjam/core/rulewrite/`**: The ONE rule-write lifecycle every rule-writing analyzer shares (`plan.py` list, `apply.py` stage/check/apply/undo, `store.py` staging + gzip backups, `types.py` shapes). Pure domain — no `tokenjam.cli`/`tokenjam.api` imports. Reuses `core/summarize/apply`'s guard model and `relearn_apply`'s block renderer rather than reimplementing either; the store persists rendered OUTPUT, never a recipe, so what a reviewer approved in the diff is byte-for-byte what apply writes. Surfaced by `tj rules` and `/api/v1/rules/*`. `core/optimize/rule_placement.py` is the WHERE half — named that, not `placement`, because `placement` is already a registered analyzer name for an unrelated question (Batch API lane, `analyzers/batch_placement.py`; Critical Rule 19).
 - **`tokenjam/mcp/server.py`**: FastMCP stdio server exposing observability data to Claude Code (plus the summarize tools — `list_summarize_candidates`, `summarize_prep`, `summarize_check`, `summarize_apply`, `summarize_undo`; see `core/summarize/`). Uses either a read-only DuckDB connection or HTTP proxy to `tj serve`. Initialized via `init()` from `cmd_mcp.py`.
 - **`tokenjam/cli/main.py`**: Root Click group with global options (`--config`, `--json`, `--no-color`, `--db`, `--agent`, `-v`). Registers all subcommands.
 
@@ -248,13 +277,15 @@ Post-ingest hooks run after each span is written to DB:
 
 - **`tj summarize`** (`cmd_summarize.py`) — structure-aware prompt summarization (advisory; new feature). `list` scans for prompt files worth summarizing (catalog-default; any scope-widening flag opens it to `*.md`) and estimates the per-call token saving — read-only. `prep`/`check` are the mechanism: `prep` wraps a prompt's structure behind id'd `<tj-keep>` markers and emits it for rewriting (CLI manual/copy, or the MCP `summarize_prep`/`summarize_check` tools for in-session rewrites); `check` re-reads + hash-guards the file, restores every block verbatim by id, and stages the result **only if structure survives** (a hard gate). No scratch state — the file on disk is the source of truth; `summarize` is in `no_db_commands` (config-only). `apply` writes a staged rewrite back to the file — default **dry-run** (it prints the unified diff), `--go` writes — guarded by an owner + content-hash + symlink check and a gzip backup; `undo` restores from that backup (refusing on drift). `prep --via claude-p` (your local `claude -p`, no key) or `prep --via api` (Anthropic with your `TJ_ANTHROPIC_API_KEY` + the required `[summarize] api_model`, reporting a "pays for itself" amortization) runs the rewrite for you in one shot — both still pass the `check` gate before staging.
 
+- **`tj rules`** (`cmd_rules.py`, `core/rulewrite/`) — the permanent CLAUDE.md rules the analyzers propose, and WHERE each one goes. `list`/`show` read the proposal cache the optimize pass already wrote (`rules` is in `no_db_commands` — config only, no analyzer sweep, works while `tj serve` holds the DuckDB lock); `stage` renders one diff PER DESTINATION; `check` re-verifies each against the file as it stands now; `apply` writes (default **dry-run**, `--go` writes, gzip backup first); `undo` reverts per destination; `applied` lists what can still be undone. The safety model is `core/summarize/apply`'s, reused rather than reimplemented (owner + content-hash + symlink guards, atomic write, refuse-on-drift undo), and the block rendering is `relearn_apply.render_note_content`'s, so a re-apply replaces its own marked block instead of duplicating it and the existing Revert path can still find it. **Partial outcomes are first-class:** three project files written and a fourth skipped because it drifted is the reported result, never rounded up to "applied". Mirrored at `GET/POST /api/v1/rules/*` and on the Lens **Optimize ▸ Rules** screen (`#/optimize/rules`), which exists because a Review-inbox card can express one write and cannot express "this rule, into these 4 of your 11 projects, here is each diff, apply selectively".
+
 - **`tj pricing list`** (`cmd_pricing.py`) — read-only inspection of the resolved model pricing table. Prints one row per `(provider, model)` with `input` / `output` / `cache_read` / `cache_write` rates and a `source` column (`override` / `packaged`) showing which layer the rate resolved from. Supports `--json` and `--model <substring>` (case-insensitive filter). `pricing` is in `no_db_commands` (config-only). The source derivation lives in `core/pricing.load_pricing_sources()` (public) — the CLI reads it rather than re-deriving precedence; a listed row is always `packaged` or `override` (the `DEFAULT_INPUT_PER_MTOK` / `DEFAULT_OUTPUT_PER_MTOK` fallback in `core/pricing.py` applies only to models absent from the table, which never appear here). The `set` half of a future `tj pricing set|list` is not built yet (#282 shipped `list` only).
 
 All commands support `--json` for machine-readable output. Commands that query alerts use exit code 1 if active (unacknowledged, unsuppressed) alerts exist.
 
 **CLI testing pattern:** Tests use `click.testing.CliRunner` with `unittest.mock.patch` on `tokenjam.cli.main.load_config` and `tokenjam.cli.main.open_db` to inject an `InMemoryBackend` and test config. See `tests/integration/test_cli.py`. Note: `cmd_doctor` opens its own DuckDB connection via `config.storage.path` to verify writability — in tests you must set this to a real temp path (e.g. `tmp_path / "test.duckdb"`).
 
-**`no_db_commands` in `cli/main.py`:** Commands that don't open the DB at startup — currently `{stop, uninstall, onboard, mcp, demo, policy, proxy, summarize, pricing}`. New commands that read only from config (or do their own DB connection later) should be added to this set so they work when `tj serve` holds the write lock. Tests for these commands can patch `open_db` with `side_effect=AssertionError(...)` to verify they never touch the DB.
+**`no_db_commands` in `cli/main.py`:** Commands that don't open the DB at startup — read the set in `cli/main.py` rather than trusting an enumeration here. New commands that read only from config (or do their own DB connection later) should be added to this set so they work when `tj serve` holds the write lock. Tests for these commands can patch `open_db` with `side_effect=AssertionError(...)` to verify they never touch the DB.
 
 **Test factories:** `tests/factories.py` provides `make_llm_span(billing_account="anthropic", ...)` and `make_session(plan_tier="api", ...)` with safe defaults that preserve existing test behavior. Tests exercising subscription / local / unknown plan-tier rendering paths should pass the field explicitly.
 
@@ -288,7 +319,7 @@ That routing lives behind **one seam, not per-command duck-typing**: `resolve_da
 
 ### Web UI ("TokenJam Lens")
 
-`tokenjam/ui/index.html` is the served dashboard — a **single-file Preact + htm SPA** (no build step, no TypeScript, no client-side router). "TokenJam Lens" is the **brand only**: it appears in `<title>`, the sidebar wordmark, and the OpenAPI title, but never in module names, route paths, or config keys. Screens: **Overview** (the default landing route — a triage front door), Status, Traces, Cost, Alerts, Drift, Optimize, Budget.
+`tokenjam/ui/index.html` is the served dashboard — a **single-file Preact + htm SPA** (no build step, no TypeScript, no client-side router). "TokenJam Lens" is the **brand only**: it appears in `<title>`, the sidebar wordmark, and the OpenAPI title, but never in module names, route paths, or config keys. Screens: **Overview** (the default landing route — a triage front door), Status, Traces, Cost, Alerts, Drift, Optimize (with **Summarize** and **Rules** sub-views), Budget.
 
 - **Offline-first (Critical Rule 18):** every JS/CSS dep is vendored under `tokenjam/ui/vendor/` — Preact + hooks + htm (ESM via `<script type="importmap">`) and **uPlot** (vendored IIFE global `uPlot` + CSS, pinned in `docs/internal/lens-vendor-versions.md`). No render-time external HTTP. `tests/unit/test_ui_offline.py` enforces this; clickable `<a href>` links are the only allowed external URLs.
 - **Single compute path:** the UI reads everything from the REST API and **never re-implements analysis, aggregation, or plan-tier framing in JS** — it consumes the `framing` block (see `core/framing.py`). If the UI needs a number, extend the endpoint; don't compute it client-side. This applies to gating/filter sets too, not just numbers: if a Python-side map decides which analyzers/findings apply to a persona (e.g. `core/optimize/runner.py`'s `PERSONA_DISABLED_ANALYZERS`), the API already publishes the resolved set (`persona_disabled_analyzers` on `/optimize`) — read that field everywhere the gate is needed, never re-declare the map as a second JS literal. Two call sites deriving the same set from two different sources will silently desync the moment either side edits its copy.
@@ -356,6 +387,10 @@ When a span has a `conversation_id` matching an existing session, it's attribute
 
 38. **Do not assume a telemetry source carries an id for the thing it describes — check, then derive the key from stored COLUMNS when it does not.** Cross-observer accounting needs a name for the underlying API call, and the obvious design is to stamp the provider's own response id on both sides. That works wherever the observer sees the provider's response — `sdk/integrations/anthropic.py` stamps `gen_ai.response.id`, `core/backfill.py` stamps `tj.call_id` off the assistant message key. It does not work for the pairing that matters most on this product's dominant corpus: **Claude Code's own OTel log exporter emits no per-call identifier at all.** Read `ClaudeCodeEvents` in `otel/semconv.py` for what its `api_request` event actually carries, and confirm there before designing around an id — the absence is the kind of premise that silently invalidates a whole approach. A stamp only ONE side of a pair can carry is worse than no stamp: it yields two different keys for one call, so the pairing never fires and nothing looks broken. **Derive the cross-observer key from columns the store already holds rather than from an attribute an ingest path promises to write** — that also makes it readable on rows written before any stamping existed, which is what lets `tj doctor --repair` clean a database an older build already doubled. Keep stamping the ids that DO exist: they are exact where a derived key is only strong, and they are what a future exporter revision would let you key on directly. If you are about to rely on the absence, re-verify the exporter's attribute set rather than trusting this line.
 
+39. **A flag set UPSTREAM of a pass that rewrites the same field is not a flag, it is a suggestion — and a unit test on the constructor cannot see it.** An analyzer correctly set `advisory_only` on a family and cleared its write offer at construction; a later pass, `_apply_write_budget`, then assigned `write_offered` from an allocator that knew nothing about the flag and quietly restored the offer. Every value on the field was individually correct at the moment it was written, and the last writer won. The unit test passed the whole time because it exercised the CONSTRUCTOR path directly while the daemon ran the COMPOSED path — the test was structurally incapable of observing the defect, so a green suite was evidence of nothing. **The fix shape is not a second guard in the later pass** (that is a race between two writers, which the next pass re-opens); it is to **remove the item from the INPUT the later pass consumes**, so no decision exists to overwrite. Here the advisory family never becomes a `WriteCandidate` at all. Two rules follow. *(a) When you set a field that a downstream pass also writes, either withdraw the row from that pass's input or move the decision into it — never both write it.* Grep for other assignments to the field before assuming your assignment is the final one. *(b) A test that constructs the object under test proves the constructor works and says nothing about the pipeline.* Assert through the composed path a user actually reaches (build the report, run the budget pass, then read the field), which is the same lesson Critical Rule 36 draws for repair passes.
+
+40. **A fixture that is a real domain object can stop being a valid vehicle for what it tests while every assertion still passes — for the wrong reason.** Eight fixtures seeded one particular relearn failure family in order to exercise write-budget behaviour. That family later became advisory-only, so it can no longer demonstrate ANY budget behaviour at all: it is never offered, never priced, never ranked. The tests stayed green, because the assertions they made happened to hold vacuously on a family that had left the population under test. This happened twice on the same branch, with two different families. The hazard is specific to fixtures that are real domain objects rather than stubs: **changing a domain object's semantics silently invalidates every test that used it as scaffolding, and the suite reports success either way.** Three habits. *(a) When you change what a family, analyzer or kind MEANS, grep the tests for its name and re-read each use — asking not "does this still pass" but "can this fixture still exhibit the behaviour the test claims to check".* *(b) Prefer a fixture that would FAIL loudly if it stopped qualifying* — assert the precondition explicitly (`assert cluster.write_offered`) before asserting the behaviour, so the vehicle's own validity is pinned rather than assumed. *(c) When you do have to swap the fixture, say in the docstring which object it replaced and why the old one stopped qualifying* — that note is what stops the next reader from "restoring" it.
+
 ## Config
 
 Config is TOML, discovered at: `tj.toml` -> `.tj/config.toml` -> `~/.config/tj/config.toml`. Override with `--config` or `TJ_CONFIG` env var. Full config hierarchy is in `tokenjam/core/config.py` (`TjConfig` dataclass).

{tokenjam-0.6.2 → tokenjam-0.6.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: tokenjam
-Version: 0.6.2
+Version: 0.6.3
 Summary: TokenJam: a local-first, OTel-native cost-saving utility for AI agents
 Project-URL: Homepage, https://tokenjam.dev
 Project-URL: Repository, https://github.com/Metabuilder-Labs/tokenjam

{tokenjam-0.6.2 → tokenjam-0.6.3}/docs/getting-started.md

@@ -2,15 +2,15 @@
 
 The fastest way onto TokenJam is to install it and run the onboarding wizard for your agent; that
 wires up everything (statusline, backfill, dashboard) in one step. This page walks every entry path
-by persona, most-common first; each rung ends with a way to verify it actually worked before you move
+by persona, most-common first; each step ends with a way to verify it actually worked before you move
 on.
 
 > **Just looking first?** `npx tokenjam` prints a 15-second read-only report over the Claude Code logs
 > you already have on disk (`~/.claude/projects/*.jsonl`); nothing installed, nothing written to disk,
-> no daemon. You should see a "Where your quota goes" panel with a percentage breakdown (or "No Claude
-> Code logs found," which just means you haven't run Claude Code on this machine yet). See
+> no daemon. You should see one line telling you how much of your recent sessions was avoidable (or
+> "No Claude Code logs found," which just means you haven't run Claude Code on this machine yet). See
 > [docs/installation.md](installation.md) for the `uv`/`pipx` runner requirements. When you're ready to
-> keep it, start at rung 1 below.
+> keep it, start at step 1 below.
 
 ## 1. Claude Code / Codex onboarding wizards
 

{tokenjam-0.6.2 → tokenjam-0.6.3}/npm-wrapper/package.json

@@ -1,6 +1,6 @@
 {
   "name": "tokenjam",
-  "version": "0.6.2",
+  "version": "0.6.3",
   "description": "Zero-install launcher for TokenJam (tj): npx tokenjam runs the Python CLI via uvx/pipx and reports the recurring mistakes your AI agent keeps repeating, no setup required.",
   "keywords": ["claude-code", "ccusage", "tokens", "cost-optimization", "llm", "agents", "observability", "tokenjam", "anthropic", "claude", "token-usage", "agent-behavior", "cli", "statusline", "opentelemetry", "agent-observability"],
   "homepage": "https://tokenjam.dev",

{tokenjam-0.6.2 → tokenjam-0.6.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "tokenjam"
-version = "0.6.2"
+version = "0.6.3"
 description = "TokenJam: a local-first, OTel-native cost-saving utility for AI agents"
 readme = "README.md"
 requires-python = ">=3.10"

{tokenjam-0.6.2 → tokenjam-0.6.3}/sdk-ts/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@tokenjam/sdk",
-  "version": "0.6.2",
+  "version": "0.6.3",
   "description": "TypeScript SDK for TokenJam — local-first observability for AI agents",
   "main": "dist/index.js",
   "types": "dist/index.d.ts",

{tokenjam-0.6.2 → tokenjam-0.6.3}/tests/integration/test_cli.py

@@ -12,7 +12,7 @@ from click.testing import CliRunner
 
 from tokenjam.cli.main import cli
 from tokenjam.core.config import AgentConfig, BudgetConfig, TjConfig
-from tokenjam.core.db import InMemoryBackend
+from tokenjam.core.db import SPANS_INDEX_SQL, InMemoryBackend
 from tokenjam.core.models import (
     AgentRecord,
     Alert,
@@ -473,6 +473,12 @@ def _duckdb_with_dropped_migration_7(tmp_path):
         backend.conn.execute(f"DROP INDEX IF EXISTS {idx}")
     backend.conn.execute("ALTER TABLE spans DROP COLUMN request_params")
     backend.conn.execute("ALTER TABLE spans DROP COLUMN request_tools")
+    # DuckDB refuses to drop a column while the table carries ART indexes, so
+    # the indexes above come off first — and have to go straight back on. Left
+    # off, this fixture would additionally reproduce the SEPARATE fault doctor's
+    # index-integrity check exists for, and these tests would be asserting an
+    # exit code that has nothing to do with the missing column they are about.
+    backend.conn.execute(SPANS_INDEX_SQL)
     return backend
 
 

{tokenjam-0.6.2 → tokenjam-0.6.3}/tests/integration/test_cost_proposals_api.py

@@ -11,6 +11,8 @@ from datetime import timedelta
 import httpx
 import pytest
 
+from tokenjam.core.rulewrite.kinds import DELIVERY_SKILL
+
 from tokenjam.api.app import create_app
 from tokenjam.core.config import ApiAuthConfig, ApiConfig, StorageConfig, TjConfig
 from tokenjam.core.db import InMemoryBackend
@@ -198,7 +200,7 @@ async def test_apply_workspace_refuses_a_caller_supplied_proposed_fix(
 
 
 async def test_cost_apply_workspace_writes_note_and_records_marker(app, client, db, monkeypatch, tmp_path):
-    """A CC-origin subagent proposal routes a reversible rung-1 note through the
+    """A CC-origin subagent proposal routes a reversible CLAUDE.md rule through the
     existing relearn apply path, then records the cost marker for delta-verify."""
     from tokenjam.core.optimize import relearn_apply as pa
 
@@ -253,17 +255,17 @@ async def test_cost_apply_workspace_writes_note_and_records_marker(app, client,
     assert any(r["analyzer"] == "subagent" for r in applied["applied"])
 
 
-async def test_cost_apply_workspace_writes_skill_for_a_rung2_proposal_and_reverts(
+async def test_cost_apply_workspace_writes_skill_for_a_skill_proposal_and_reverts(
     app, client, config, db, monkeypatch, tmp_path,
 ):
     """`apply-workspace` is generic across analyzers, not special-cased to
-    `subagent`: a rung-2 skill-note proposal routes through the SAME path,
+    `subagent`: a skill-note proposal routes through the SAME path,
     writes, and reverts cleanly.
 
     Two facts are pinned here. First, a deterministic tool-call cluster on a
     claude-code window produces NO `script` card any more: the analyzer has no
     fix that survives for that persona and is skipped before it runs. Second,
-    the generic rung-2 write/revert route still works, exercised by seeding the
+    the generic skill write/revert route still works, exercised by seeding the
     proposal the adapter builds directly into the store."""
     from tokenjam.core.optimize import relearn_apply as pa, relearn_store
     from tokenjam.core.optimize.analyzers.workflow_restructure import (
@@ -322,7 +324,7 @@ async def test_cost_apply_workspace_writes_skill_for_a_rung2_proposal_and_revert
     prop = script_props[0]
     assert prop["apply_capable"] is True
     assert prop["advise_only"] is False
-    assert prop["rung"] == 2
+    assert prop["delivery"] == DELIVERY_SKILL
 
     body = {"proposal_id": prop["proposal_id"], "target_path": str(target)}
 

{tokenjam-0.6.2 → tokenjam-0.6.3}/tests/integration/test_db.py

@@ -212,11 +212,76 @@ def test_delete_spans_before_cutoff(db):
     db.insert_span(span_new)
 
     cutoff = now - timedelta(days=90)
-    deleted = db.delete_spans_before(cutoff)
-    assert deleted == 1
+    spans_deleted, sessions_deleted = db.delete_spans_before(cutoff)
+    assert spans_deleted == 1
 
     remaining = db.conn.execute("SELECT COUNT(*) FROM spans").fetchone()
     assert remaining[0] == 1
+    # The session straddles the cutoff and still has a live span, so it stays:
+    # a session is orphaned by the delete only once it has NO spans left.
+    assert sessions_deleted == 0
+    assert db.conn.execute("SELECT COUNT(*) FROM sessions").fetchone()[0] == 1
+
+
+def test_delete_spans_before_cutoff_takes_the_sessions_it_orphans(db):
+    """Deleting only spans left the parent sessions asserting a day had data.
+
+    `core/data_span` unions `sessions.started_at` into the day set it measures
+    the available span from, so every orphan went on claiming a day carried data
+    after the data for that day was destroyed — the deletion skewed the measure
+    of what survived it.
+    """
+    _insert_agent(db)
+    now = utcnow()
+    old = now - timedelta(days=100)
+
+    aged_out = make_session(started_at=old, ended_at=old + timedelta(minutes=1))
+    db.upsert_session(aged_out)
+    db.insert_span(make_llm_span(session_id=aged_out.session_id, start_time=old))
+
+    live = make_session()
+    db.upsert_session(live)
+    db.insert_span(make_llm_span(
+        session_id=live.session_id, start_time=now - timedelta(days=1),
+    ))
+
+    spans_deleted, sessions_deleted = db.delete_spans_before(now - timedelta(days=90))
+    assert (spans_deleted, sessions_deleted) == (1, 1)
+
+    surviving = [
+        r[0] for r in db.conn.execute("SELECT session_id FROM sessions").fetchall()
+    ]
+    assert surviving == [live.session_id]
+
+
+def test_delete_spans_before_cutoff_takes_an_aged_out_session_with_no_spans(db):
+    """A pre-cutoff session with no spans is aged-out history like any other.
+
+    Left behind it would go on asserting, to `core/data_span`, that a day
+    beyond the retention horizon carried data.
+    """
+    _insert_agent(db)
+    now = utcnow()
+    empty = make_session(
+        started_at=now - timedelta(days=100),
+        ended_at=now - timedelta(days=100) + timedelta(minutes=1),
+    )
+    db.upsert_session(empty)
+
+    spans_deleted, sessions_deleted = db.delete_spans_before(now - timedelta(days=90))
+    assert (spans_deleted, sessions_deleted) == (0, 1)
+    assert db.conn.execute("SELECT COUNT(*) FROM sessions").fetchone()[0] == 0
+
+
+def test_delete_spans_before_cutoff_keeps_a_live_session_with_no_spans_yet(db):
+    """An open session that has not written a span is not aged out."""
+    _insert_agent(db)
+    now = utcnow()
+    db.upsert_session(make_session(started_at=now, ended_at=None))
+
+    spans_deleted, sessions_deleted = db.delete_spans_before(now - timedelta(days=90))
+    assert (spans_deleted, sessions_deleted) == (0, 0)
+    assert db.conn.execute("SELECT COUNT(*) FROM sessions").fetchone()[0] == 1
 
 
 # -- Traces --

{tokenjam-0.6.2 → tokenjam-0.6.3}/tests/integration/test_inbox_headline_total_api.py

@@ -18,6 +18,8 @@ from datetime import datetime, timedelta, timezone
 import httpx
 import pytest
 
+from tokenjam.core.rulewrite.kinds import DELIVERY_CLAUDE_MD_RULE
+
 from tokenjam.api.app import create_app
 from tokenjam.core.config import ApiAuthConfig, ApiConfig, StorageConfig, TjConfig
 from tokenjam.core.db import InMemoryBackend
@@ -67,6 +69,18 @@ def app(config, db):
             cache_tokens=400, session_id=f"s-{i}",
             start_time=now - timedelta(days=2, minutes=i),
         ))
+    # The corpus has to be 30 days WIDE, not merely 30 days' worth of rows. The
+    # inbox window is the analysis span bounded by how far back this store's
+    # oldest row actually sits, so a two-day corpus resolves to a two-day
+    # window — and the hand-built relearn buckets below, which are the point of
+    # these tests, are labelled `30d`. One row at the far edge makes the two
+    # sides name the same window without changing what any analyzer finds.
+    db.insert_span(make_llm_span(
+        agent_id="svc-a", provider="anthropic", model="claude-sonnet-5",
+        billing_account="anthropic", input_tokens=15_000, output_tokens=200,
+        cache_tokens=400, session_id="s-oldest",
+        start_time=now - timedelta(days=29),
+    ))
     pipeline = IngestPipeline(db=db, config=config)
     return create_app(config=config, db=db, ingest_pipeline=pipeline)
 
@@ -103,7 +117,7 @@ def _cluster(
 ) -> RelearnCluster:
     return RelearnCluster(
         signature=signature, family_key=signature, title=f"{signature} recurs",
-        sessions=3, occurrences=9, repos=["demo"], rung=1, scope="project",
+        sessions=3, occurrences=9, repos=["demo"], delivery=DELIVERY_CLAUDE_MD_RULE, scope="project",
         proposed_fix="Do the thing that stops this.",
         past_overspend_usd=unbounded_usd, past_overspend_tokens=unbounded_tokens,
         past_reread_usd=round(unbounded_usd * 0.1, 6),

{tokenjam-0.6.2 → tokenjam-0.6.3}/tests/integration/test_relearn_api.py

@@ -4,7 +4,7 @@
   1. Mutating endpoints (apply/enable/disable/revert/refresh) require the
      always-on local write token, independent of ``api.auth.enabled``.
   2. ``/apply`` refuses a ``target_path`` outside the user's home directory
-     (defense-in-depth allowlist) and — for rung 1 — a target that isn't an
+     (defense-in-depth allowlist) and — for a CLAUDE.md rule — a target that isn't an
      allowlisted note file (see test_relearn_apply.py for the relearn_apply
      unit-level version of that same guard).
 
@@ -16,6 +16,8 @@ from __future__ import annotations
 import httpx
 import pytest
 
+from tokenjam.core.rulewrite.kinds import DELIVERY_CLAUDE_MD_RULE, DELIVERY_INJECTING_HOOK
+
 from tokenjam.api.app import create_app
 from tokenjam.core.config import ApiAuthConfig, ApiConfig, StorageConfig, TjConfig
 from tokenjam.core.db import InMemoryBackend
@@ -72,7 +74,7 @@ def stored_proposal(config) -> str:
     cluster = RelearnCluster(
         signature="cwd_confusion", family_key="cwd_confusion",
         title="cwd / relative-path confusion", sessions=5, occurrences=9,
-        repos=["demo"], rung=1, scope="project",
+        repos=["demo"], delivery=DELIVERY_CLAUDE_MD_RULE, scope="project",
         proposed_fix="Verify an absolute cwd before a relative Read.",
     )
     relearn_store.write_cache(RelearnFinding(clusters=[cluster]), config=config)
@@ -212,7 +214,7 @@ async def test_apply_allows_target_inside_home(app, client, monkeypatch, tmp_pat
     assert r.json()["dry_run"] is False
 
 
-# --- must-fix #2 (routed through the API): rung-1 note target allowlist -------
+# --- must-fix #2 (routed through the API): note target allowlist -------------
 
 async def test_apply_note_route_refuses_non_markdown_target(
     app, client, monkeypatch, tmp_path, stored_proposal,
@@ -257,7 +259,7 @@ def _store_cost_proposal(config, **overrides) -> str:
         "target_key": {}, "evidence": "", "baseline": {},
         "advise_text": "Route explore to the cheaper same-family model.",
         "proposed_fix": "Route explore to the cheaper same-family model.",
-        "rung": 0, "scope": "project", "apply_capable": True,
+        "delivery": "", "scope": "project", "apply_capable": True,
         "apply_kind": "agent_model", "agent_name": "explore",
         "current_model": "claude-opus-4-8", "proposed_model": "claude-haiku-4-5",
     }
@@ -405,7 +407,7 @@ async def test_apply_refuses_a_stored_proposal_missing_a_required_field(
     assert target.read_text() == _AGENT_FILE
 
 
-async def test_rung_ladder_apply_opens_no_cost_window(
+async def test_delivery_apply_opens_no_cost_window(
     app, client, config, monkeypatch, tmp_path, stored_proposal,
 ):
     """A plain note fix has no priced metric, so it must not create a cost
@@ -463,7 +465,8 @@ async def test_apply_rejects_a_client_constructed_cluster_payload(
     target.write_text("# Repo\n", encoding="utf-8")
 
     body = _apply_body(str(target), proposal_id=stored_proposal)
-    body.update({"signature": "attacker", "rung": 3, "title": "not from the detector",
+    body.update({"signature": "attacker", "delivery": DELIVERY_INJECTING_HOOK,
+                 "title": "not from the detector",
                  "proposed_fix": "rm -rf /"})
     r = await client.post(
         "/api/v1/relearn/apply", json=body,
@@ -607,7 +610,7 @@ async def test_proposals_flag_example_sessions_that_resolve(config, db, client):
     cluster = RelearnCluster(
         signature="cwd_confusion", family_key="cwd_confusion",
         title="cwd / relative-path confusion", sessions=2, occurrences=3,
-        repos=["demo"], rung=1, scope="project",
+        repos=["demo"], delivery=DELIVERY_CLAUDE_MD_RULE, scope="project",
         proposed_fix="Verify an absolute cwd before a relative Read.",
         examples=[
             RelearnExample(session_id="ingested-1", repo="demo", ts=None, snippet="a"),
@@ -641,7 +644,7 @@ async def test_advise_only_proposals_carry_their_reason_in_the_payload(config, c
     advise = RelearnCluster(
         signature="http_call:peer closed", family_key=None,
         title="peer closed the connection", sessions=3, occurrences=9,
-        repos=["billing-svc"], rung=1, scope="project",
+        repos=["billing-svc"], delivery=DELIVERY_CLAUDE_MD_RULE, scope="project",
         proposed_fix="Retry the upstream call with backoff.",
         examples=[RelearnExample(session_id="s1", repo="billing-svc", ts=None, snippet="e")],
         advise_only=True,
@@ -649,7 +652,7 @@ async def test_advise_only_proposals_carry_their_reason_in_the_payload(config, c
     workspace = RelearnCluster(
         signature="cwd_confusion", family_key="cwd_confusion",
         title="cwd / relative-path confusion", sessions=4, occurrences=12,
-        repos=["demo"], rung=1, scope="project",
+        repos=["demo"], delivery=DELIVERY_CLAUDE_MD_RULE, scope="project",
         proposed_fix="Verify an absolute cwd before a relative Read.",
     )
     relearn_store.write_cache(RelearnFinding(clusters=[advise, workspace]), config=config)

{tokenjam-0.6.2 → tokenjam-0.6.3}/tests/integration/test_relearn_window_api.py

@@ -35,6 +35,26 @@ from tests.factories import make_session
 ANCHOR = datetime(2026, 7, 27, 12, 0, tzinfo=timezone.utc)
 
 
+def utc_noon(days_ago: int) -> datetime:
+    """A seed instant that cannot straddle a day boundary, in any timezone.
+
+    The day-span measure counts DISTINCT UTC days and discards anything dated
+    later than today, so a fixture seeded at an offset from ``utcnow()`` puts
+    its rows wherever the wall clock happens to be: rows land on one calendar
+    day for part of the day and two for the rest, and a row seeded at "now" can
+    read as tomorrow under a database timezone that runs ahead of UTC. Both
+    made the assertions below go red for a few hours out of every day and green
+    again afterwards, which reads as "whoever pushed last broke it".
+
+    Noon UTC is the furthest any instant can be from both boundaries, so the
+    day a row belongs to is fixed no matter when or where the suite runs.
+    """
+    today = datetime.now(tz=timezone.utc).date()
+    return datetime(
+        today.year, today.month, today.day, 12, 0, tzinfo=timezone.utc,
+    ) - timedelta(days=days_ago)
+
+
 @pytest.fixture
 def db():
     backend = InMemoryBackend()
@@ -203,7 +223,9 @@ async def test_a_cache_without_windowed_figures_says_so_rather_than_faking_one(
 @pytest.mark.asyncio
 async def test_the_days_of_data_available_measure_is_served(client, config, db):
     _seed_relearn_cache(config)
-    db.upsert_session(make_session(session_id="s1", agent_id="claude-code-x"))
+    db.upsert_session(make_session(
+        session_id="s1", agent_id="claude-code-x", started_at=utc_noon(0),
+    ))
     async with client as c:
         body = (await c.get("/api/v1/relearn/proposals")).json()
     span = body["data_span"]
@@ -220,8 +242,9 @@ async def test_an_ancient_row_does_not_inflate_the_served_span(client, config, d
     actually carries data on.
     """
     _seed_relearn_cache(config)
-    now = datetime.now(tz=timezone.utc)
-    db.upsert_session(make_session(session_id="s1", agent_id="a", started_at=now))
+    db.upsert_session(make_session(
+        session_id="s1", agent_id="a", started_at=utc_noon(0),
+    ))
     db.upsert_session(make_session(
         session_id="ancient", agent_id="a",
         started_at=datetime(2020, 1, 1, tzinfo=timezone.utc),