claude-context-cost 1.3.1__tar.gz

claude_context_cost-1.3.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Peter
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

claude_context_cost-1.3.1/PKG-INFO

@@ -0,0 +1,200 @@
+Metadata-Version: 2.4
+Name: claude-context-cost
+Version: 1.3.1
+Summary: How much each source (CLAUDE.md, rules, skills, memory, tool results) costs in the context of a Claude Code session, counted from the API usage in its transcript.
+Author: Peter
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/peterwkdev-creator/claude-context-cost
+Project-URL: Issues, https://github.com/peterwkdev-creator/claude-context-cost/issues
+Keywords: claude-code,context-window,tokens,transcript,cli
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# claude-context-cost
+
+How much each source (your `CLAUDE.md`, rules, skills, memory, tool results)
+costs in the context of a Claude Code session, counted from the API `usage`
+recorded in its transcript, not estimated from characters.
+
+![The page counts the synthetic example, then a transcript file](docs/demo.gif)
+
+**[Try it in the browser](https://peterwkdev-creator.github.io/claude-context-cost/)**:
+choose one of your transcripts, or run the synthetic example. The library runs
+in the page (Pyodide); your file is never uploaded.
+
+The usual estimate, characters / 4, **undercounts by 40 to 45%**: −45% over
+5,365 steps of the author's sessions, mostly in Portuguese, and −40% in an
+English-language session (2026-10-10). Here every response adds up to its
+`usage` exactly, in all 457 transcripts tested.
+
+> Reads only the transcripts under `~/.claude/projects` on your machine.
+> Nothing is sent anywhere. The numbers are tokens, not dollars.
+
+## Install
+
+```bash
+pip install claude-context-cost
+```
+
+Python 3.11 or newer, no dependencies. To install a tagged version straight
+from GitHub instead:
+
+```bash
+pip install "git+https://github.com/peterwkdev-creator/claude-context-cost@v1.3.1"
+```
+
+## Usage
+
+```bash
+claude-context-cost --example
+```
+
+```bash
+claude-context-cost ~/.claude/projects/<project-folder>/<session-id>.jsonl
+```
+
+The newest `.jsonl` in a project's folder is its latest session (on Windows,
+`~` is `%USERPROFILE%`). A subagent has its own transcript,
+`<session-id>/subagents/agent-<id>.jsonl`.
+
+Pass a project's folder instead of a file to sum its sessions and their
+subagents: the costliest sources first, then one line per session. Pass
+`~/.claude/projects` itself to sum every project. Tools and your own
+`~/.claude/...` files add up across projects; a project's `CLAUDE.md` and
+rules stay apart, marked with the project.
+
+`--steps` shows what entered before each response, `--all` lists every
+source, `--json` is for scripts (for a folder too). On the synthetic example
+shipped with the package:
+
+```text
+        cost  share   entered    in    chars/4  error  how                  source
+     128,240  80.6%    36,640     7          -      -  -                    unattributed: system prompt, tools, overhead
+      10,400   6.5%     3,200     4      2,000   -38%  measured, split      tool: Read
+       5,600   3.5%     1,600     7      1,000   -38%  estimated            claude.md: CLAUDE.md
+       1,600   1.0%       800     4        500   -38%  split, estimated     path rule: .claude/rules/python.md
+```
+
+- **cost**: tokens × the responses the source stayed in context, until a
+  compaction removed it. This is what you pay for over the session.
+- **entered**: the tokens it brought in; **in**: the responses it stayed for.
+- **chars/4** and **error**: what the usual estimate would say, beside it.
+
+From Python:
+
+```python
+from claude_context_cost import analyze, example_path
+
+report = analyze(example_path())  # or the path of any .jsonl transcript
+print(report.total)  # 159160: input tokens summed over the 7 responses
+read = report.sources[("tool", "Read")]
+print(read.entered, read.responses, read.cost)  # 3200 4 10400
+print(read.estimate_div4)  # 2000.0: characters / 4 for the same text
+```
+
+`summarize(folder)` returns the same sum as a folder on the command line:
+`.sources` maps `(kind, name, project)` to the totals, with `project` set only
+for a project's own files.
+
+## Why
+
+[claude-devtools](https://github.com/matt1398/claude-devtools) already shows
+the context by source. Read in its code at commit `1486f20` (the latest on
+2026-10-10):
+
+- Each source's tokens are characters / 4 (`src/main/utils/tokenizer.ts`);
+  only the turn total comes from the `usage`.
+- Rules in `.claude/rules/` are listed as global sources without reading
+  their `paths:`, so a rule that never loaded still counts. Issue
+  [#224](https://github.com/matt1398/claude-devtools/issues/224), open since
+  2026-07-17.
+- `CLAUDE.md` and rules are read from disk today, not as they were during the
+  session.
+
+This tool fixes the first two from the transcript alone.
+
+## What it counts
+
+- **One response per `message.id`.** A response is written over several
+  lines with the same `usage`; it counts once.
+- **Context = `input_tokens` + `cache_read_input_tokens` +
+  `cache_creation_input_tokens`.**
+- **What entered** between two responses is the context growth minus the
+  previous response's output. When a single source entered, its tokens are
+  **measured**. When several did, the growth is **split** by their size.
+- **At the start of the session and after each compaction** the context
+  jumps at once: each source is **estimated** with the bytes-per-token ratio
+  measured in the same transcript (median of its text-only steps; 2.25 when
+  it has fewer than 3), and the rest is **unattributed** (system prompt,
+  tool definitions). So every response adds up to its `usage`.
+- **Rules with `paths:`** count only from the response where Claude Code
+  loaded them (`nested_memory`), never before.
+- **Only what Claude Code rendered** into the context counts; bookkeeping
+  lines in the transcript do not.
+- **A compaction** removes everything; messages it keeps count as "kept by
+  compaction", not as new responses.
+
+## How it is tested
+
+- A synthetic transcript with known values (`tests/fixtures/build_example.py`)
+  where every source's tokens are worked out by hand: each one comes out
+  exact, every response adds up to its `usage`, a `paths:` rule that never
+  loaded counts 0, a response written over several lines counts once.
+- `python tests/mutate.py` breaks each of those rules in a copy of the code
+  (17 mutants); the tests catch all 17.
+- On the author's machine (2026-10-10): 75 session transcripts and 382
+  subagent transcripts, 91,347 responses, 0 where the sources differ from the
+  `usage`, 0 where the response count differs from an independent count, and
+  the sum over every project takes every transcript. The
+  ratio was measured in 50 of the 73 sessions with responses, median 2.22
+  bytes per token. Run it on yours with
+  `CLAUDE_CONTEXT_COST_CORPUS=1 python -m unittest tests.test_corpus -v`
+  (it prints only counts).
+
+## Limits
+
+- **Split and estimated tokens are not measured.** The `how` column says
+  which is which. Many steps bring a tool result together with a small
+  reminder, so splitting is common.
+- **Output includes thinking.** Thinking may leave the context later, so
+  the previous output can be overcounted; the remainder then shows as
+  negative unattributed tokens (191 of 91,347 responses).
+- **A session's total does not include its subagents**; the sum of its
+  project's folder does.
+- **English was measured in one session** (2026-10-10): 8 file reads, a
+  license, release notes, two READMEs, three Python modules and a git log;
+  median −40%, from −29% to −47%. The other numbers come from one person's
+  sessions, mostly in Portuguese.
+- The transcript format is not documented and changes between versions.
+
+## Compatibility
+
+Transcripts of Claude Code 2.1 (the synthetic fixture follows 2.1.293).
+Python 3.11 to 3.14, tested on Linux and Windows.
+
+Versions follow [Semantic Versioning](https://semver.org/): the names in
+`claude_context_cost.__all__` and the keys of `--json` change incompatibly only
+in a new major version. The transcript format is Claude Code's and changes
+without notice; when it does, a fix keeps the API and ships as a minor or
+patch version.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md). Never paste a real transcript in an
+issue or pull request: it holds paths, file contents and sometimes secrets.
+
+## License
+
+[MIT](LICENSE)

claude_context_cost-1.3.1/README.md

@@ -0,0 +1,176 @@
+# claude-context-cost
+
+How much each source (your `CLAUDE.md`, rules, skills, memory, tool results)
+costs in the context of a Claude Code session, counted from the API `usage`
+recorded in its transcript, not estimated from characters.
+
+![The page counts the synthetic example, then a transcript file](docs/demo.gif)
+
+**[Try it in the browser](https://peterwkdev-creator.github.io/claude-context-cost/)**:
+choose one of your transcripts, or run the synthetic example. The library runs
+in the page (Pyodide); your file is never uploaded.
+
+The usual estimate, characters / 4, **undercounts by 40 to 45%**: −45% over
+5,365 steps of the author's sessions, mostly in Portuguese, and −40% in an
+English-language session (2026-10-10). Here every response adds up to its
+`usage` exactly, in all 457 transcripts tested.
+
+> Reads only the transcripts under `~/.claude/projects` on your machine.
+> Nothing is sent anywhere. The numbers are tokens, not dollars.
+
+## Install
+
+```bash
+pip install claude-context-cost
+```
+
+Python 3.11 or newer, no dependencies. To install a tagged version straight
+from GitHub instead:
+
+```bash
+pip install "git+https://github.com/peterwkdev-creator/claude-context-cost@v1.3.1"
+```
+
+## Usage
+
+```bash
+claude-context-cost --example
+```
+
+```bash
+claude-context-cost ~/.claude/projects/<project-folder>/<session-id>.jsonl
+```
+
+The newest `.jsonl` in a project's folder is its latest session (on Windows,
+`~` is `%USERPROFILE%`). A subagent has its own transcript,
+`<session-id>/subagents/agent-<id>.jsonl`.
+
+Pass a project's folder instead of a file to sum its sessions and their
+subagents: the costliest sources first, then one line per session. Pass
+`~/.claude/projects` itself to sum every project. Tools and your own
+`~/.claude/...` files add up across projects; a project's `CLAUDE.md` and
+rules stay apart, marked with the project.
+
+`--steps` shows what entered before each response, `--all` lists every
+source, `--json` is for scripts (for a folder too). On the synthetic example
+shipped with the package:
+
+```text
+        cost  share   entered    in    chars/4  error  how                  source
+     128,240  80.6%    36,640     7          -      -  -                    unattributed: system prompt, tools, overhead
+      10,400   6.5%     3,200     4      2,000   -38%  measured, split      tool: Read
+       5,600   3.5%     1,600     7      1,000   -38%  estimated            claude.md: CLAUDE.md
+       1,600   1.0%       800     4        500   -38%  split, estimated     path rule: .claude/rules/python.md
+```
+
+- **cost**: tokens × the responses the source stayed in context, until a
+  compaction removed it. This is what you pay for over the session.
+- **entered**: the tokens it brought in; **in**: the responses it stayed for.
+- **chars/4** and **error**: what the usual estimate would say, beside it.
+
+From Python:
+
+```python
+from claude_context_cost import analyze, example_path
+
+report = analyze(example_path())  # or the path of any .jsonl transcript
+print(report.total)  # 159160: input tokens summed over the 7 responses
+read = report.sources[("tool", "Read")]
+print(read.entered, read.responses, read.cost)  # 3200 4 10400
+print(read.estimate_div4)  # 2000.0: characters / 4 for the same text
+```
+
+`summarize(folder)` returns the same sum as a folder on the command line:
+`.sources` maps `(kind, name, project)` to the totals, with `project` set only
+for a project's own files.
+
+## Why
+
+[claude-devtools](https://github.com/matt1398/claude-devtools) already shows
+the context by source. Read in its code at commit `1486f20` (the latest on
+2026-10-10):
+
+- Each source's tokens are characters / 4 (`src/main/utils/tokenizer.ts`);
+  only the turn total comes from the `usage`.
+- Rules in `.claude/rules/` are listed as global sources without reading
+  their `paths:`, so a rule that never loaded still counts. Issue
+  [#224](https://github.com/matt1398/claude-devtools/issues/224), open since
+  2026-07-17.
+- `CLAUDE.md` and rules are read from disk today, not as they were during the
+  session.
+
+This tool fixes the first two from the transcript alone.
+
+## What it counts
+
+- **One response per `message.id`.** A response is written over several
+  lines with the same `usage`; it counts once.
+- **Context = `input_tokens` + `cache_read_input_tokens` +
+  `cache_creation_input_tokens`.**
+- **What entered** between two responses is the context growth minus the
+  previous response's output. When a single source entered, its tokens are
+  **measured**. When several did, the growth is **split** by their size.
+- **At the start of the session and after each compaction** the context
+  jumps at once: each source is **estimated** with the bytes-per-token ratio
+  measured in the same transcript (median of its text-only steps; 2.25 when
+  it has fewer than 3), and the rest is **unattributed** (system prompt,
+  tool definitions). So every response adds up to its `usage`.
+- **Rules with `paths:`** count only from the response where Claude Code
+  loaded them (`nested_memory`), never before.
+- **Only what Claude Code rendered** into the context counts; bookkeeping
+  lines in the transcript do not.
+- **A compaction** removes everything; messages it keeps count as "kept by
+  compaction", not as new responses.
+
+## How it is tested
+
+- A synthetic transcript with known values (`tests/fixtures/build_example.py`)
+  where every source's tokens are worked out by hand: each one comes out
+  exact, every response adds up to its `usage`, a `paths:` rule that never
+  loaded counts 0, a response written over several lines counts once.
+- `python tests/mutate.py` breaks each of those rules in a copy of the code
+  (17 mutants); the tests catch all 17.
+- On the author's machine (2026-10-10): 75 session transcripts and 382
+  subagent transcripts, 91,347 responses, 0 where the sources differ from the
+  `usage`, 0 where the response count differs from an independent count, and
+  the sum over every project takes every transcript. The
+  ratio was measured in 50 of the 73 sessions with responses, median 2.22
+  bytes per token. Run it on yours with
+  `CLAUDE_CONTEXT_COST_CORPUS=1 python -m unittest tests.test_corpus -v`
+  (it prints only counts).
+
+## Limits
+
+- **Split and estimated tokens are not measured.** The `how` column says
+  which is which. Many steps bring a tool result together with a small
+  reminder, so splitting is common.
+- **Output includes thinking.** Thinking may leave the context later, so
+  the previous output can be overcounted; the remainder then shows as
+  negative unattributed tokens (191 of 91,347 responses).
+- **A session's total does not include its subagents**; the sum of its
+  project's folder does.
+- **English was measured in one session** (2026-10-10): 8 file reads, a
+  license, release notes, two READMEs, three Python modules and a git log;
+  median −40%, from −29% to −47%. The other numbers come from one person's
+  sessions, mostly in Portuguese.
+- The transcript format is not documented and changes between versions.
+
+## Compatibility
+
+Transcripts of Claude Code 2.1 (the synthetic fixture follows 2.1.293).
+Python 3.11 to 3.14, tested on Linux and Windows.
+
+Versions follow [Semantic Versioning](https://semver.org/): the names in
+`claude_context_cost.__all__` and the keys of `--json` change incompatibly only
+in a new major version. The transcript format is Claude Code's and changes
+without notice; when it does, a fix keeps the API and ships as a minor or
+patch version.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md). Never paste a real transcript in an
+issue or pull request: it holds paths, file contents and sometimes secrets.
+
+## License
+
+[MIT](LICENSE)

claude_context_cost-1.3.1/claude_context_cost/__init__.py

@@ -0,0 +1,15 @@
+"""How much each source costs in the context of a Claude Code session, counted from the usage."""
+from pathlib import Path
+
+from .attribute import Report, Source, analyze, attribute
+from .summary import Summary, Total, summarize
+from .transcript import read
+
+__version__ = "1.3.1"
+__all__ = ["Report", "Source", "analyze", "attribute", "example_path", "read", "Summary", "summarize",
+           "Total", "__version__"]
+
+
+def example_path():
+    """A synthetic transcript shipped with the package, for trying the tool without a real one."""
+    return Path(__file__).resolve().parent / "example.jsonl"

claude_context_cost-1.3.1/claude_context_cost/__main__.py

@@ -0,0 +1,5 @@
+import sys
+
+from .cli import main
+
+sys.exit(main())

claude_context_cost-1.3.1/claude_context_cost/attribute.py

@@ -0,0 +1,175 @@
+"""Gives every token of every response's input context to a source.
+
+Between two responses of the same segment, the tokens that entered are known
+exactly from the usage:
+
+    entered(n) = context(n) - context(n-1) - output(n-1)
+
+One item in between: it gets all of them ("measured"). Several: they share
+them in proportion to their bytes ("split"). At the first response of a
+segment (session start or right after a compaction) the context also holds
+the system prompt and the tools, which are not in the transcript: each item
+gets its bytes divided by the transcript's own bytes-per-token ratio
+("estimated") and the remainder is unattributed.
+
+So in every response the sources plus the unattributed add up to the usage,
+and the cost of a source is its tokens summed over every response it stayed
+in, until a compaction drops it.
+"""
+import statistics
+from dataclasses import dataclass, field
+
+from .transcript import read
+
+# Measured in October 2026 on 1,831 pairs from 12 projects (mostly Portuguese):
+# 2.25 bytes per token. Used only when a transcript has too few pairs of its own.
+DEFAULT_BYTES_PER_TOKEN = 2.25
+MIN_PAIR_BYTES = 2000
+MAX_PAIR_OUTPUT = 400   # the previous output includes thinking that may not come back
+MIN_PAIRS = 3
+
+OUTPUT = ("output", "responses")
+FLOOR = ("unattributed", "system prompt, tools, overhead")
+
+
+@dataclass
+class Source:
+    kind: str
+    name: str
+    entered: int = 0     # tokens that entered the context, summed over every time it entered
+    bytes: int = 0
+    chars: int = 0
+    opaque: int = 0      # items whose size is not in the text (images, tool references)
+    cost: int = 0       # tokens x responses it stayed in
+    responses: int = 0
+    how: dict = field(default_factory=dict)
+
+    @property
+    def estimate_div4(self):
+        """What a characters / 4 estimate would say it entered, or None if it has no text."""
+        return self.chars / 4 if self.chars and not self.opaque else None
+
+
+@dataclass
+class Report:
+    path: str
+    responses: list
+    compactions: int
+    bad_lines: int
+    bytes_per_token: float
+    ratio_pairs: int
+    ratio_is_default: bool
+    sources: dict
+
+    @property
+    def total(self):
+        return sum(r.context for r in self.responses)
+
+
+def allocate(total, weights):
+    """Integers proportional to the weights that add up to total exactly (largest remainder)."""
+    s = sum(weights)
+    if not weights or s <= 0:
+        return [0] * len(weights)
+    raw = [total * w / s for w in weights]
+    out = [int(x // 1) for x in raw]
+    order = sorted(range(len(raw)), key=lambda i: raw[i] - out[i], reverse=True)
+    for i in order[:total - sum(out)]:
+        out[i] += 1
+    return out
+
+
+def calibrate(responses):
+    """Bytes per token from the steps where only text entered: the median, or the default."""
+    ratios = []
+    for prev, r in zip(responses, responses[1:]):
+        if r.segment_start or not r.entered or any(i.opaque for i in r.entered):
+            continue
+        delta = r.context - prev.context - prev.output
+        size = sum(i.bytes for i in r.entered)
+        if size >= MIN_PAIR_BYTES and prev.output < MAX_PAIR_OUTPUT and delta > 0:
+            ratios.append(size / delta)
+    if len(ratios) >= MIN_PAIRS:
+        return statistics.median(ratios), len(ratios), False
+    return DEFAULT_BYTES_PER_TOKEN, len(ratios), True
+
+
+def _estimated(items, budget, ratio):
+    """Segment start: bytes / ratio for each item, never more than the budget; returns the rest."""
+    known = [i for i in items if not i.opaque]
+    est = round(sum(i.bytes for i in known) / ratio)
+    share = allocate(min(est, max(budget, 0)), [i.bytes for i in known])
+    for i, t in zip(known, share):
+        i.tokens, i.how = t, "estimated"
+    for i in items:
+        if i.opaque:
+            i.tokens, i.how = 0, "unknown"
+    return budget - sum(share)
+
+
+def _split(items, delta, ratio):
+    """Between two responses: all of delta goes to the items; returns what is left unattributed."""
+    if delta <= 0:
+        for i in items:
+            i.tokens, i.how = 0, "shrank"
+        return delta
+    if len(items) == 1:
+        items[0].tokens, items[0].how = delta, "measured"
+        return 0
+    known = [i for i in items if not i.opaque]
+    opaque = [i for i in items if i.opaque]
+    if opaque:
+        text = min(round(sum(i.bytes for i in known) / ratio), delta)
+        shares = allocate(text, [i.bytes for i in known]) + allocate(delta - text, [1] * len(opaque))
+        ordered = known + opaque
+    else:
+        shares, ordered = allocate(delta, [i.bytes for i in known]), known
+    for i, t in zip(ordered, shares):
+        i.tokens, i.how = t, "split"
+    return delta - sum(shares)
+
+
+def attribute(transcript):
+    ratio, pairs, default = calibrate(transcript.responses)
+    sources = {}
+
+    def source(key):
+        if key not in sources:
+            sources[key] = Source(*key)
+        return sources[key]
+
+    prev, ledger = None, {}
+    for r in transcript.responses:
+        if r.segment_start or prev is None:
+            ledger = {}
+            rest = _estimated(r.entered, r.context, ratio)
+        else:
+            ledger = dict(ledger)
+            ledger[OUTPUT] = ledger.get(OUTPUT, 0) + prev.output
+            source(OUTPUT).entered += prev.output
+            rest = _split(r.entered, r.context - prev.context - prev.output, ratio)
+        for i in r.entered:
+            ledger[i.key] = ledger.get(i.key, 0) + i.tokens
+            s = source(i.key)
+            s.entered += i.tokens
+            s.bytes += i.bytes
+            s.chars += i.chars
+            s.opaque += i.opaque
+            s.how[i.how] = s.how.get(i.how, 0) + 1
+        if rest:
+            ledger[FLOOR] = ledger.get(FLOOR, 0) + rest
+            source(FLOOR).entered += rest
+        ledger = {k: v for k, v in ledger.items() if v}
+        r.ledger = ledger
+        for k, v in ledger.items():
+            source(k).cost += v
+            source(k).responses += 1
+        prev = r
+    sources = {k: s for k, s in sources.items() if s.entered or s.cost}
+    return Report(transcript.path, transcript.responses, transcript.compactions, transcript.bad_lines,
+                  ratio, pairs, default, sources)
+
+
+def analyze(path):
+    """Read one transcript and attribute its context. See `Report`."""
+    return attribute(read(path))