tierview 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +190 -2
- package/config/default-rules.yml +276 -0
- package/config/estimates.json +24 -0
- package/config/prices.json +35 -0
- package/dist/bin.js +2486 -0
- package/package.json +61 -4
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Mohamed Elsaka
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,3 +1,191 @@
|
|
|
1
|
-
|
|
1
|
+
<div align="center">
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
# tierview
|
|
4
|
+
|
|
5
|
+
**Tiered Claude code review with a cost receipt**
|
|
6
|
+
|
|
7
|
+
Caught 9 of 13 real bugs vs 7 for all-Opus, with zero false alarms, at 16% lower cost.<br>
|
|
8
|
+
<sub>(13 held-out bugs, not statistically significant; see <a href="bench/results/RESULTS.md">RESULTS.md</a>)</sub>
|
|
9
|
+
|
|
10
|
+

|
|
11
|
+
|
|
12
|
+
<sub>`tierview review --dry-run` on the 4-file change in <a href="demo/make-sample-repo.mjs"><code>demo/</code></a>: it shows where every hunk would go and what the review would cost, with zero model calls. Recorded with <a href="demo/demo.tape">VHS</a>.</sub>
|
|
13
|
+
|
|
14
|
+
</div>
|
|
15
|
+
|
|
16
|
+

|
|
17
|
+
|
|
18
|
+
## What it does
|
|
19
|
+
|
|
20
|
+
tierview reads a diff, scores each hunk for risk with deterministic rules (auth, SQL, memory, concurrency, boundary changes and so on), and sends it to the cheapest Claude model that can handle it: boring hunks to Sonnet 5.5 at low effort, risky ones to Opus 5.5. When a model is unsure, contradicts itself or reports something serious, the hunk goes up a tier for a second look, and a higher tier can refute a lower tier's finding. At the end it prints a **cost receipt**: tokens and cost per tier, and what the same work would have cost on the most expensive model.
|
|
21
|
+
|
|
22
|
+
### A real receipt
|
|
23
|
+
|
|
24
|
+
This is a saved run from the benchmark ([`django-lookup-rhs-iterator__tierview.json`](bench/results/runs/django-lookup-rhs-iterator__tierview.json)), a Django commit whose later fix says it consumed iterator right-hand sides. tierview caught it. The explanation is shortened here; the receipt is as printed.
|
|
25
|
+
|
|
26
|
+
```text
|
|
27
|
+
tierview: 1 finding(s) in 4 reviewed hunk(s).
|
|
28
|
+
|
|
29
|
+
[medium] django/db/models/lookups.py:290-304 self.rhs iterated twice; one-shot iterables (generators) lose values
|
|
30
|
+
bug, confidence 0.45, tier 3 claude-opus-5-5
|
|
31
|
+
If rhs is a one-shot iterator, any() uses it up and the loop then gets nothing, so the
|
|
32
|
+
In lookup matches nothing and the query silently returns no rows. (shortened)
|
|
33
|
+
fix: Turn the iterable into a list once before checking it, e.g. `rhs = list(self.rhs)`.
|
|
34
|
+
|
|
35
|
+
Receipt (claude-code backend: 2 claude call(s); cost is API-equivalent at list prices as of 2026-11-16, not what your plan bills)
|
|
36
|
+
tier model effort hunks +escalated calls input cache write cache read output via Claude Code direct API
|
|
37
|
+
1 claude-sonnet-5-5 low 3 0 1 2 13,900 994 297 $0.06 $0.03
|
|
38
|
+
2 claude-opus-5-5 medium 0 0 0 0 0 0 0 $0.00 $0.00
|
|
39
|
+
3 claude-opus-5-5 high 1 0 1 2 6,406 994 4,315 $0.14 $0.11
|
|
40
|
+
|
|
41
|
+
total: $0.20 via Claude Code, $0.14 direct API (without Claude Code's per-call overhead) (26,910 tokens)
|
|
42
|
+
all-claude-fable-5-2: $0.51 via Claude Code, $0.33 direct API (the same tokens at its rates) -> 61.5% less
|
|
43
|
+
skipped hunks: 0 not reviewed: 0 wall time: 62.4s
|
|
44
|
+
Tier 3 ran on Opus 5.5 (Fable not enabled)
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
The all-Fable line is a **calculation** (this run's tokens priced at Fable 5.2 list rates), not a measurement: no Fable call was made, and a real all-Fable run would send different tokens. On a Claude plan the dollar figures are API-equivalent; the plan billed nothing extra.
|
|
48
|
+
|
|
49
|
+
## Install
|
|
50
|
+
|
|
51
|
+
### Free path: your Claude subscription, no API key
|
|
52
|
+
|
|
53
|
+
tierview's default backend runs the official [Claude Code](https://code.claude.com) CLI (`claude -p`) once per batch of hunks, so it uses the plan you already pay for. It never reads or copies Claude Code's login; it only runs the binary.
|
|
54
|
+
|
|
55
|
+
```bash
|
|
56
|
+
# 1. Install and sign in to Claude Code (once): https://code.claude.com
|
|
57
|
+
# 2. Then, in any git repo with changes:
|
|
58
|
+
npx tierview review --dry-run # free: zero model calls, shows routing and an estimate
|
|
59
|
+
npx tierview review # the real review, on your plan
|
|
60
|
+
```
|
|
61
|
+
|
|
62
|
+
Or install it: `npm install -g tierview`. Needs Node 22 or newer.
|
|
63
|
+
|
|
64
|
+
Plans have usage limits. The benchmark ran 190 calls on a Pro plan in batches of at most 20 and still hit the 5-hour limit once (see [RESULTS.md](bench/results/RESULTS.md)). `--max-calls <n>` and `--max-cost <usd>` cap a run, and a capped run says exactly which hunks it left unreviewed (exit code 3).
|
|
65
|
+
|
|
66
|
+
### With an API key
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
export ANTHROPIC_API_KEY=...
|
|
70
|
+
npx tierview review --backend api
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The `api` backend bills your key at list prices. It is unit-tested against a mocked SDK; it has not been run live.
|
|
74
|
+
|
|
75
|
+
## How the tiers work
|
|
76
|
+
|
|
77
|
+
```text
|
|
78
|
+
diff
|
|
79
|
+
|
|
|
80
|
+
v
|
|
81
|
+
Tier 0 parse hunks, score risk with rules (free, no model)
|
|
82
|
+
| generated files (lockfiles, dist/) are skipped
|
|
83
|
+
|
|
|
84
|
+
+-- score < 25 -------> Tier 1 Sonnet 5.5, low effort
|
|
85
|
+
+-- score 25 to 59 ---> Tier 2 Opus 5.5, medium effort (with Fable off, Tiers 2
|
|
86
|
+
+-- score >= 60 ------> Tier 3 Opus 5.5, high effort and 3 run as one Opus call)
|
|
87
|
+
|
|
88
|
+
Tier 1 hunk goes up when: confidence < 0.7, the model asks for help,
|
|
89
|
+
it reports a high-severity finding, it contradicts itself, or the call fails.
|
|
90
|
+
A higher tier verifies or refutes lower-tier findings, then:
|
|
91
|
+
v
|
|
92
|
+
merge duplicates, rank by severity, print findings + cost receipt
|
|
93
|
+
```
|
|
94
|
+
|
|
95
|
+
Routing costs nothing and is inspectable: `--dry-run` prints the points behind every hunk's tier (for example `auth +35, sql +30, boundary +25`). Weights and thresholds are data in [`config/default-rules.yml`](config/default-rules.yml); every key is in [docs/CONFIG.md](docs/CONFIG.md).
|
|
96
|
+
|
|
97
|
+
### Fable is opt-in
|
|
98
|
+
|
|
99
|
+
On some Claude plans, Fable 5.2 uses paid extra usage, and `claude -p` does not ask before using it. So on the default backend tierview never requests Fable unless you ask:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
tierview review --allow-fable # or allowFable: true in .tierview.yml
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
With the flag it prints a warning before the first call. Without it, Tier 3 runs on Opus 5.5 and the receipt says so. **Fable was never measured in the benchmark**, so nothing here supports a claim about its quality.
|
|
106
|
+
|
|
107
|
+
### `--dry-run`
|
|
108
|
+
|
|
109
|
+
`tierview review --dry-run` makes zero model calls. It prints each hunk's score, tier and reasons, an estimated cost per tier, the all-Fable comparison and the savings estimate (see the GIF above). The token estimates are rough placeholders calibrated on a handful of runs; treat the dollar figures as an order of magnitude.
|
|
110
|
+
|
|
111
|
+
## Other ways to run it
|
|
112
|
+
|
|
113
|
+
```bash
|
|
114
|
+
tierview review --staged # staged changes only
|
|
115
|
+
tierview review --base main --head HEAD # a range
|
|
116
|
+
tierview review --diff-file change.diff # a saved diff (no file context)
|
|
117
|
+
tierview review --json --sarif out.sarif # machine-readable output
|
|
118
|
+
tierview review --fail-on high # exit 1 on findings at or above high
|
|
119
|
+
tierview pr https://github.com/OWNER/REPO/pull/123 --dry-run # a pull request (set GITHUB_TOKEN)
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
### GitHub Action
|
|
123
|
+
|
|
124
|
+
[`examples/tierview.yml`](examples/tierview.yml) is a ready workflow. Add your own `ANTHROPIC_API_KEY` as a repository secret; the Action reviews each pull request on the API backend, posts one summary comment (findings and receipt, updated in place) and stops at `max-cost` (default $2.00 per run). Fork PRs, which get no secrets, are skipped with a notice.
|
|
125
|
+
|
|
126
|
+
**Not tested live yet.** The Action and its PR comment have been tested only with mocked GitHub responses. They have not run on a real runner against a real PR. CI for this repo runs the typecheck, the tests, the build and a packed-package smoke test on Ubuntu, Windows and macOS with Node 22 and 24, and all of that is green, but that is not the same as the Action working end to end. Treat the first runs as a trial and [open an issue](https://github.com/moelsaka01/tierview/issues) if it breaks.
|
|
127
|
+
|
|
128
|
+
## Benchmark
|
|
129
|
+
|
|
130
|
+
On a held-out test split of **13 buggy and 7 clean commits** from 9 open-source repos (QEMU, DPDK, Django, Cilium, Tetragon, Firecracker, Next.js and others), each run once per arm on the `claude-code` backend. The bugs are real: each commit introduced a bug that a later commit fixed and named. Config frozen before the test run (tag `bench-config-v2`).
|
|
131
|
+
|
|
132
|
+
| arm | bugs caught (95% range) | clean cases with a false alarm | cost per case via Claude Code | cost per bug caught |
|
|
133
|
+
|---|---|---:|---:|---:|
|
|
134
|
+
| **tierview** | **9 / 13 = 69%** (46–92%) | **0 / 7** | $0.213 | $0.364 |
|
|
135
|
+
| all-Opus | 7 / 13 = 54% (23–85%) | 3 / 7 | $0.254 | $0.525 |
|
|
136
|
+
| all-Sonnet | 4 / 13 = 31% (8–54%) | 0 / 7 | $0.113 | $0.408 |
|
|
137
|
+
|
|
138
|
+
### What this does and does not show
|
|
139
|
+
|
|
140
|
+
- **The difference is not statistically significant.** tierview caught 2 bugs all-Opus missed; all-Opus caught none that tierview missed. Exact McNemar p = 0.5. The 95% ranges overlap for every pair of arms. This is a pilot, not proof.
|
|
141
|
+
- **n = 13.** And C-heavy: 7 of the 13 test bugs are C (QEMU, DPDK); TypeScript has one. It says little about other languages.
|
|
142
|
+
- **tierview cost about 1.9x all-Sonnet.** It is not the cheapest arm, only the cheapest of the two that caught most bugs.
|
|
143
|
+
- **One run per case and arm.** Models vary between runs; single runs cannot measure that.
|
|
144
|
+
- **The clean set is older and calmer** (2025 commits with no fix touching them for a year), so false-alarm rates may be optimistic.
|
|
145
|
+
- **The judge is mine to audit, not independent.** A rule plus an Opus fallback decides whether a finding matches the known bug. I hand-checked 20 judgements: 14 agree, 0 disagree, 6 unsure. The marker was the author.
|
|
146
|
+
- **The dev split disagreed on all-Sonnet** (it looked best there), which is why the test split is the headline and dev was used only for tuning.
|
|
147
|
+
- **No Fable arm.** Costs for Fable are calculated, not measured.
|
|
148
|
+
|
|
149
|
+
Everything, including every case, the raw runs, the judgements and a usage-limit incident that was found and fixed before any metric was computed, is in [bench/results/RESULTS.md](bench/results/RESULTS.md). Reproduce with `npm run bench:build` and the commands at the bottom of that file (the full run is 190 `claude` calls).
|
|
150
|
+
|
|
151
|
+
## How it compares
|
|
152
|
+
|
|
153
|
+
Facts as checked on 2026-11-16; products change, so check their pages.
|
|
154
|
+
|
|
155
|
+
| | routes by | escalates | cost shown | published benchmark of the router | license |
|
|
156
|
+
|---|---|---|---|---|---|
|
|
157
|
+
| **tierview** | each hunk, with deterministic rules | yes, on confidence, contradiction and severity | per-tier receipt with an all-Fable counterfactual | yes, small (this repo) | MIT |
|
|
158
|
+
| [d1ff](https://github.com/d1ff-dev/d1ff) | each change into simple / medium / complex model tiers | not found | not found | has a `benchmark/` directory, no results found | MIT |
|
|
159
|
+
| OrcaCode Review | cheap model screens, strong model does a final pass | not found | not found | not found | not checked |
|
|
160
|
+
| Qodo (PR-Agent core is Apache-2.0) | review depth per PR, chosen by a small LLM classifier | not found | not found | not found | commercial product; open core |
|
|
161
|
+
| CodeRabbit | closed | not found | per-seat pricing | not found | closed |
|
|
162
|
+
|
|
163
|
+
Tiered routing on its own is not new, and tierview is not the first tiered reviewer. What I did not find elsewhere is confidence-based escalation between tiers, a cost receipt with a counterfactual, and a published ground-truth benchmark of the router against single-model baselines. "Not found" means I did not find it in two searches and two page reads, not that it does not exist.
|
|
164
|
+
|
|
165
|
+
## FAQ
|
|
166
|
+
|
|
167
|
+
**Is it really free?** There is no tierview fee and no API key on the default path; it spends your existing Claude plan's usage. `--dry-run` spends nothing. A real review makes about 2 `claude` calls (1.9 on average in the benchmark), and plans have limits.
|
|
168
|
+
|
|
169
|
+
**Does it send my code anywhere?** To Anthropic, through Claude Code (or the API, with `--backend api`), exactly as if you pasted the diff into Claude. tierview itself has no server and sends nothing else. On the default backend the `claude` process runs in an empty temp directory in safe mode, so the reviewed repo's CLAUDE.md, hooks and MCP servers never load.
|
|
170
|
+
|
|
171
|
+
**Why not just use the best model on everything?** You can: the all-Opus arm is that. On this small test it cost 16% more per case and was no better (7 vs 9, within noise) and raised false alarms on 3 of 7 clean commits. With 13 bugs, I cannot tell you that the cascade is better, only that I did not see it be worse.
|
|
172
|
+
|
|
173
|
+
**Does the cost receipt show what I was billed?** On a plan, no: it is the API-equivalent cost of the tokens that ran, at the list prices in [`config/prices.json`](config/prices.json) (with an `as_of` date). Your plan billed nothing extra. On `--backend api` it is what your key was charged at list prices.
|
|
174
|
+
|
|
175
|
+
**Can I make it review more or less carefully?** Yes: `.tierview.yml` sets thresholds, per-path `min_tier` and `max_tier`, signal weights and models. See [docs/CONFIG.md](docs/CONFIG.md). Weights and thresholds were tuned on the 10-case dev split only.
|
|
176
|
+
|
|
177
|
+
**What does it miss?** 4 of the 13 test bugs, 3 C and 1 Rust: an int overflow, an ignored PTE bit, an address-space layout problem and a thread-startup race. A bug in a boring-looking hunk gets Sonnet at low effort; escalation is the mitigation, not a guarantee.
|
|
178
|
+
|
|
179
|
+
**Which Claude models does it use?** Sonnet 5.5 and Opus 5.5 by default, Fable 5.2 only if you opt in. The IDs are in `config/default-rules.yml`.
|
|
180
|
+
|
|
181
|
+
## Development
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
npm install
|
|
185
|
+
npm run check # typecheck + tests (no network, no model calls)
|
|
186
|
+
npm run build
|
|
187
|
+
```
|
|
188
|
+
|
|
189
|
+
[PLAN.md](PLAN.md) is the design spec and [BUILD_LOG.md](BUILD_LOG.md) records how it was built, including what went wrong. [CHANGELOG.md](CHANGELOG.md) lists releases. Ideas for a hosted version are in [docs/HOSTED.md](docs/HOSTED.md) (not built).
|
|
190
|
+
|
|
191
|
+
MIT licensed.
|
|
@@ -0,0 +1,276 @@
|
|
|
1
|
+
# tierview default rules. A repo's .tierview.yml is merged over this file.
|
|
2
|
+
# Spec: PLAN.md §4. Patterns are JavaScript regexes, matched case-insensitively.
|
|
3
|
+
# path -> tested against the file path (forward slashes)
|
|
4
|
+
# content -> tested against added and removed lines (without the +/- prefix)
|
|
5
|
+
# content_added -> added lines only
|
|
6
|
+
# content_removed -> removed lines only
|
|
7
|
+
# logic -> implemented in src/tier0/signals.ts, no regex
|
|
8
|
+
# Score = sum over categories of the largest matching weight in the category
|
|
9
|
+
# + structure points - reductions, clamped to 0..100.
|
|
10
|
+
|
|
11
|
+
version: 1
|
|
12
|
+
|
|
13
|
+
models:
|
|
14
|
+
tier1: { model: claude-sonnet-5-5, effort: low }
|
|
15
|
+
tier2: { model: claude-opus-5-5, effort: medium }
|
|
16
|
+
tier3: { model: claude-fable-5-2, effort: high }
|
|
17
|
+
|
|
18
|
+
thresholds: { tier2: 25, tier3: 60 } # Phase 4 (P4-09, dev split): tier2 30 -> 25
|
|
19
|
+
|
|
20
|
+
escalation:
|
|
21
|
+
tier1_min_confidence: 0.7
|
|
22
|
+
tier2_min_confidence: 0.6
|
|
23
|
+
tier1_verify_severity: high # findings at or above this go up a tier for confirmation
|
|
24
|
+
tier2_verify_severity: critical
|
|
25
|
+
max_rate: 0.3
|
|
26
|
+
|
|
27
|
+
budget: { max_cost_usd: null }
|
|
28
|
+
|
|
29
|
+
# File context for Tier 1 (PLAN §5): none = the diff's own 3 lines; tier3 = the same as Tier 3 (whole file when short,
|
|
30
|
+
# else ±80 lines and the enclosing function). bench-config-v2 (Phase 4, decided from dev results): tier3.
|
|
31
|
+
tier1_context: tier3
|
|
32
|
+
|
|
33
|
+
# The counterfactual every receipt prices: every reviewed hunk sent to this model as a Tier 3 request.
|
|
34
|
+
receipt:
|
|
35
|
+
baseline: { model: claude-fable-5-2, effort: high }
|
|
36
|
+
|
|
37
|
+
# Fable on the claude-code backend is opt-in (PLAN.md §5a). On some plans Fable uses paid extra usage and
|
|
38
|
+
# `claude -p` does not ask first. false: tiers set to Fable run on the Tier 2 model; --allow-fable overrides.
|
|
39
|
+
allowFable: false
|
|
40
|
+
|
|
41
|
+
# Which backend runs the models. Spec: PLAN.md §5a.
|
|
42
|
+
backend:
|
|
43
|
+
default: claude-code # claude-code | api (--backend overrides)
|
|
44
|
+
claude_code:
|
|
45
|
+
binary: null # path to the official `claude` binary; null = PATH, then ~/.local/bin
|
|
46
|
+
max_hunks_per_call: 40 # hunks per `claude -p` call (all tiers batch across files)
|
|
47
|
+
max_tokens_per_call: 40000 # estimated input tokens per call
|
|
48
|
+
timeout_s: 900
|
|
49
|
+
api:
|
|
50
|
+
max_tokens: 16000
|
|
51
|
+
concurrency: 4
|
|
52
|
+
|
|
53
|
+
ignore: []
|
|
54
|
+
rules: []
|
|
55
|
+
|
|
56
|
+
file_kinds: # first match wins, in this order
|
|
57
|
+
generated:
|
|
58
|
+
- '**/package-lock.json'
|
|
59
|
+
- '**/yarn.lock'
|
|
60
|
+
- '**/pnpm-lock.yaml'
|
|
61
|
+
- '**/Cargo.lock'
|
|
62
|
+
- '**/poetry.lock'
|
|
63
|
+
- '**/uv.lock'
|
|
64
|
+
- '**/go.sum'
|
|
65
|
+
- '**/composer.lock'
|
|
66
|
+
- '**/Gemfile.lock'
|
|
67
|
+
- '**/dist/**'
|
|
68
|
+
- '**/build/**'
|
|
69
|
+
- '**/vendor/**'
|
|
70
|
+
- '**/node_modules/**'
|
|
71
|
+
- '**/*.min.js'
|
|
72
|
+
- '**/*.min.css'
|
|
73
|
+
- '**/*.map'
|
|
74
|
+
- '**/*.snap'
|
|
75
|
+
- '**/__snapshots__/**'
|
|
76
|
+
- '**/*.pb.go'
|
|
77
|
+
- '**/*_pb2.py'
|
|
78
|
+
- '**/*.generated.*'
|
|
79
|
+
docs:
|
|
80
|
+
- '**/*.md'
|
|
81
|
+
- '**/*.mdx'
|
|
82
|
+
- '**/*.rst'
|
|
83
|
+
- '**/!(requirements*).txt'
|
|
84
|
+
- 'docs/**'
|
|
85
|
+
- '**/LICENSE*'
|
|
86
|
+
- '**/CHANGELOG*'
|
|
87
|
+
test:
|
|
88
|
+
- '**/test/**'
|
|
89
|
+
- '**/tests/**'
|
|
90
|
+
- '**/__tests__/**'
|
|
91
|
+
- '**/spec/**'
|
|
92
|
+
- '**/*.test.*'
|
|
93
|
+
- '**/*.spec.*'
|
|
94
|
+
- '**/*_test.go'
|
|
95
|
+
- '**/*_test.py'
|
|
96
|
+
- '**/test_*.py'
|
|
97
|
+
- '**/*Test.java'
|
|
98
|
+
- '**/*Tests.cs'
|
|
99
|
+
config:
|
|
100
|
+
- '**/*.yml'
|
|
101
|
+
- '**/*.yaml'
|
|
102
|
+
- '**/*.toml'
|
|
103
|
+
- '**/*.ini'
|
|
104
|
+
- '**/*.json'
|
|
105
|
+
- '**/Dockerfile*'
|
|
106
|
+
- '**/*.tf'
|
|
107
|
+
|
|
108
|
+
categories:
|
|
109
|
+
secrets:
|
|
110
|
+
weight: 40
|
|
111
|
+
path:
|
|
112
|
+
- '(^|/)\.env(\.|$)'
|
|
113
|
+
- 'credentials|secrets?(\.|/|$)'
|
|
114
|
+
- '\.(pem|key|p12|pfx)$'
|
|
115
|
+
content_added:
|
|
116
|
+
- 'AKIA[0-9A-Z]{16}'
|
|
117
|
+
- '-----BEGIN (RSA |EC |OPENSSH )?PRIVATE KEY-----'
|
|
118
|
+
- '\b(api[_-]?key|secret|passwd|password|token)\b\s*[:=]\s*[''"][^''"]{8,}[''"]'
|
|
119
|
+
- '\b(sk|pk|ghp|gho|xox[bap])[-_][A-Za-z0-9_-]{16,}'
|
|
120
|
+
|
|
121
|
+
auth:
|
|
122
|
+
weight: 35
|
|
123
|
+
path:
|
|
124
|
+
- '(^|/)(auth\w*|login|logout|signin|signup|sessions?|oauth\d?|sso|saml|jwt|identity|passwords?)(/|[._-]|$)'
|
|
125
|
+
content:
|
|
126
|
+
- '\b(authenticat\w*|authoriz\w*|authn|authz)\b'
|
|
127
|
+
- '\b(jwt|oauth|saml|bearer|csrf|session[_ ]?(id|token)|refresh[_ ]?token|access[_ ]?token)\b'
|
|
128
|
+
- '\b(verify|check|compare|hash)_?(password|token|signature)\b'
|
|
129
|
+
- '\b(login|logout|sign_?in)\s*\('
|
|
130
|
+
|
|
131
|
+
crypto:
|
|
132
|
+
weight: 35
|
|
133
|
+
path:
|
|
134
|
+
- '(^|/)(crypto\w*|cipher|tls|ssl|certs?|keys?|signing)(/|[._-]|$)'
|
|
135
|
+
content:
|
|
136
|
+
- '\b(encrypt|decrypt|cipher|hmac|pbkdf2|bcrypt|scrypt|argon2|aes|rsa|ecdsa|ed25519)\w*\b'
|
|
137
|
+
- '\b(md5|sha1)\b'
|
|
138
|
+
- '\bECB\b'
|
|
139
|
+
- 'verify\s*=\s*False|rejectUnauthorized\s*:\s*false|InsecureSkipVerify\s*:\s*true'
|
|
140
|
+
- 'Math\.random\(\)|\brand\(\)|random\.random\(\)'
|
|
141
|
+
- '\b(nonce|iv|salt)\b\s*[:=]'
|
|
142
|
+
|
|
143
|
+
payment:
|
|
144
|
+
weight: 35
|
|
145
|
+
path:
|
|
146
|
+
- '(^|/)(payments?|billing|invoices?|checkout|ledger|refunds?|subscriptions?|wallet|pricing)(/|[._-]|$)'
|
|
147
|
+
content:
|
|
148
|
+
# Phase 4: "checkout" (git checkout) and "ledger" (any bookkeeping class) left the content list; they stay in path.
|
|
149
|
+
- '\b(payment|billing|invoice|refund|chargeback|stripe|paypal|braintree)\w*\b'
|
|
150
|
+
# Phase 4: a bare `total =` / `price =` fired on tierview's own estimate.ts; now a money suffix is required.
|
|
151
|
+
- '\b(amount|price|balance|total|subtotal|fee)_?(cents|minor|usd|eur|due|paid|charged)\b\s*[-+*/]?='
|
|
152
|
+
|
|
153
|
+
sql:
|
|
154
|
+
weight: 30
|
|
155
|
+
path:
|
|
156
|
+
- '\.sql$'
|
|
157
|
+
content:
|
|
158
|
+
- '\b(select|insert\s+into|update|delete\s+from)\b.{0,80}(\$\{|%s|\+\s*\w|\{\w+\}|\|\|)'
|
|
159
|
+
- '\bf[''"].{0,40}\b(select|insert|update|delete)\b'
|
|
160
|
+
- '\.(execute|executemany|raw|query|exec)\s*\(\s*(f?[''"`]|\w+\s*\+)'
|
|
161
|
+
- '\b(drop|truncate|alter)\s+table\b'
|
|
162
|
+
|
|
163
|
+
migrations:
|
|
164
|
+
weight: 30
|
|
165
|
+
path:
|
|
166
|
+
- '(^|/)(migrations?|alembic|db/migrate|prisma/migrations|flyway|liquibase)(/|$)'
|
|
167
|
+
- '(^|/)schema\.(prisma|rb|sql|graphql)$'
|
|
168
|
+
|
|
169
|
+
deserialization:
|
|
170
|
+
weight: 30
|
|
171
|
+
content:
|
|
172
|
+
- '\bpickle\.(loads?|Unpickler)\b|\bmarshal\.loads?\b|\bcPickle\b'
|
|
173
|
+
- '\byaml\.(load|unsafe_load)\s*\('
|
|
174
|
+
- '\bunserialize\s*\(|\bObjectInputStream\b|\breadObject\s*\('
|
|
175
|
+
- '(?<![.\w])(eval|exec)\s*\(|\bnew\s+Function\s*\('
|
|
176
|
+
- '\bBinaryFormatter\b|\bXMLDecoder\b'
|
|
177
|
+
|
|
178
|
+
unsafe:
|
|
179
|
+
weight: 30
|
|
180
|
+
content:
|
|
181
|
+
- '\bunsafe\b'
|
|
182
|
+
- '\b(memcpy|memmove|strcpy|strcat|sprintf|gets|alloca)\s*\('
|
|
183
|
+
- 'shell\s*=\s*True|\bos\.system\s*\(|\bchild_process\b|\bexecSync\s*\('
|
|
184
|
+
- 'dangerouslySetInnerHTML|\binnerHTML\s*=|document\.write\s*\('
|
|
185
|
+
- '\bmalloc\s*\(|\bfree\s*\(|\brealloc\s*\('
|
|
186
|
+
|
|
187
|
+
tests_weakened:
|
|
188
|
+
weight: 30
|
|
189
|
+
content_added:
|
|
190
|
+
- '\b(it|test|describe)\.(skip|only)\s*\(|\bx(it|describe|test)\s*\('
|
|
191
|
+
- '@(pytest\.mark\.)?(skip|skipif|xfail)\b|@unittest\.skip'
|
|
192
|
+
- '\bt\.Skip(f|Now)?\s*\(|#\[ignore\]|@Disabled\b|@Ignore\b'
|
|
193
|
+
logic:
|
|
194
|
+
- deleted_test_file
|
|
195
|
+
- deleted_test_function
|
|
196
|
+
|
|
197
|
+
permissions:
|
|
198
|
+
weight: 25
|
|
199
|
+
path:
|
|
200
|
+
- '(^|/)(permissions?|rbac|acl|policy|policies|roles?|guards?)(/|[._-]|$)'
|
|
201
|
+
content:
|
|
202
|
+
- '\b(is_?admin|is_?superuser|has_?(role|permission|access)|can_\w+|can(Access|Edit|Delete|View|Create|Update|Read|Write|Manage)\w*|require_?(role|permission|admin))\b'
|
|
203
|
+
- '\b(chmod|chown|sudo|setuid|setgid)\b'
|
|
204
|
+
- 'Access-Control-Allow-Origin|\bcors\b'
|
|
205
|
+
- '\b(rbac|acl|iam)\b'
|
|
206
|
+
|
|
207
|
+
concurrency:
|
|
208
|
+
weight: 25
|
|
209
|
+
content:
|
|
210
|
+
- '\b(mutex|rwlock|semaphore|spinlock|atomic\w*|synchronized|volatile)\b'
|
|
211
|
+
- '\b(lock|unlock|acquire|release)\s*\('
|
|
212
|
+
- '\b(thread|threading|goroutine|pthread_\w+|ExecutorService|asyncio|tokio::spawn)\b'
|
|
213
|
+
- '^\s*go\s+\w|<-\s*\w|\bchan\b|\bselect\s*\{'
|
|
214
|
+
- '\b(race|deadlock)\b'
|
|
215
|
+
logic:
|
|
216
|
+
- await_added_or_removed
|
|
217
|
+
|
|
218
|
+
error_handling_removed:
|
|
219
|
+
weight: 20
|
|
220
|
+
content_removed:
|
|
221
|
+
- '^\s*(try\b|catch\b|except\b|finally\b|rescue\b)'
|
|
222
|
+
- 'if\s+err\s*!=\s*nil'
|
|
223
|
+
- '^\s*(assert\b|raise\b|throw\b|panic\()'
|
|
224
|
+
- '\b(validate|sanitize|escape)\w*\s*\('
|
|
225
|
+
content_added:
|
|
226
|
+
- 'except\s*:\s*pass|catch\s*(\(\w*\))?\s*\{\s*\}'
|
|
227
|
+
- '\.unwrap\(\)|\bas\s+any\b|@ts-ignore|# type: ignore|//\s*nolint'
|
|
228
|
+
|
|
229
|
+
infra:
|
|
230
|
+
weight: 20
|
|
231
|
+
path:
|
|
232
|
+
- '(^|/)Dockerfile'
|
|
233
|
+
- '(^|/)\.github/workflows/'
|
|
234
|
+
- '\.(tf|tfvars)$'
|
|
235
|
+
- '(^|/)(k8s|kubernetes|helm|charts|deploy|terraform|ansible)(/|$)'
|
|
236
|
+
- 'docker-compose.*\.ya?ml$'
|
|
237
|
+
|
|
238
|
+
assertions_removed:
|
|
239
|
+
weight: 20
|
|
240
|
+
logic:
|
|
241
|
+
- test_assertion_count_dropped
|
|
242
|
+
|
|
243
|
+
dependencies:
|
|
244
|
+
weight: 15
|
|
245
|
+
path:
|
|
246
|
+
- '(^|/)(package\.json|requirements[^/]*\.txt|pyproject\.toml|go\.mod|Cargo\.toml|pom\.xml|build\.gradle(\.kts)?|Gemfile)$'
|
|
247
|
+
|
|
248
|
+
boundary:
|
|
249
|
+
weight: 25 # Phase 4 (P4-09, dev split): 10 -> 25
|
|
250
|
+
logic:
|
|
251
|
+
- comparison_operator_changed
|
|
252
|
+
- index_arithmetic_changed
|
|
253
|
+
- condition_changed # Phase 4: an if/while/for condition rewritten
|
|
254
|
+
- bounds_check_added # Phase 4: new `<`/`<=`/`>`/`>=` check against arithmetic or a size-like name
|
|
255
|
+
|
|
256
|
+
# Phase 4: allocation, free and object-lifetime calls (leaks, double frees, use after free).
|
|
257
|
+
memory:
|
|
258
|
+
weight: 25
|
|
259
|
+
content:
|
|
260
|
+
- '\b(malloc|calloc|realloc|free|kfree|kmalloc|kzalloc|kvfree|vmalloc|vfree|g_malloc0?|g_new0?|g_free|strdup|mmap|munmap)\s*\('
|
|
261
|
+
- '\b\w+_(alloc|allocate|zalloc|free|release|destroy|unref|put)\s*\('
|
|
262
|
+
- '\b(Box::(from_raw|into_raw|leak)|mem::(forget|transmute)|ManuallyDrop|std::alloc)\b'
|
|
263
|
+
|
|
264
|
+
structure:
|
|
265
|
+
size: [[5, 0], [30, 5], [100, 10], [.inf, 15]] # [max changed lines, points]
|
|
266
|
+
scatter: { max_files: 8, max_top_dirs: 3, points: 10 }
|
|
267
|
+
many_hunks_in_file: { more_than: 5, points: 5 }
|
|
268
|
+
churn: { enabled: true, days: 90, min_commits: 10, points: 10 }
|
|
269
|
+
hotspot: { enabled: true, days: 90, min_fix_commits: 2, points: 10, pattern: '\bfix(es|ed)?\b|bug|regress' }
|
|
270
|
+
deletion_only: { points: 5 }
|
|
271
|
+
diff_weakens_tests: { points: 10 }
|
|
272
|
+
no_tests_changed: { min_source_lines: 50, points: 5 }
|
|
273
|
+
|
|
274
|
+
reductions:
|
|
275
|
+
test_file_without_test_signal: 15
|
|
276
|
+
docs_or_trivial_cap: 5 # docs files and comment/whitespace/import-only hunks
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
{
|
|
2
|
+
"calibrated": false,
|
|
3
|
+
"note": "Partly measured in P3-10 (4 claude-code requests, 2026-11-16): per-call Claude Code overhead, system prompt size, Tier 1 and Tier 2 output. Tier 3 output on Fable is not measured yet (the live test ran Tier 3 on Opus), and n is tiny, so calibrated stays false. Phase 4 replaces these.",
|
|
4
|
+
"chars_per_token": 3,
|
|
5
|
+
"system_prompt_tokens": 1000,
|
|
6
|
+
"shared_context_tokens_per_file": 20,
|
|
7
|
+
"shared_context_tokens_per_hunk": 12,
|
|
8
|
+
"context_tokens_per_hunk": {
|
|
9
|
+
"tier1": 0,
|
|
10
|
+
"tier2": 600,
|
|
11
|
+
"tier3": 1500
|
|
12
|
+
},
|
|
13
|
+
"output_tokens_per_hunk": {
|
|
14
|
+
"tier1": 400,
|
|
15
|
+
"tier2": 700,
|
|
16
|
+
"tier3": 2000
|
|
17
|
+
},
|
|
18
|
+
"tier1_batch": {
|
|
19
|
+
"max_hunks": 12,
|
|
20
|
+
"max_tokens": 6000
|
|
21
|
+
},
|
|
22
|
+
"escalation_reserve": 0.15,
|
|
23
|
+
"claude_code_overhead_tokens_per_call": 2100
|
|
24
|
+
}
|
|
@@ -0,0 +1,35 @@
|
|
|
1
|
+
{
|
|
2
|
+
"as_of": "2026-11-16",
|
|
3
|
+
"source": "https://platform.claude.com/docs/en/about-claude/pricing",
|
|
4
|
+
"currency": "USD",
|
|
5
|
+
"unit": "per_million_tokens",
|
|
6
|
+
"models": {
|
|
7
|
+
"claude-sonnet-5-5": {
|
|
8
|
+
"input": 2.0,
|
|
9
|
+
"cache_write_5m": 2.5,
|
|
10
|
+
"cache_write_1h": 4.0,
|
|
11
|
+
"cache_read": 0.1,
|
|
12
|
+
"output": 10.0,
|
|
13
|
+
"batch_input": 1.0,
|
|
14
|
+
"batch_output": 5.0
|
|
15
|
+
},
|
|
16
|
+
"claude-opus-5-5": {
|
|
17
|
+
"input": 4.0,
|
|
18
|
+
"cache_write_5m": 5.0,
|
|
19
|
+
"cache_write_1h": 8.0,
|
|
20
|
+
"cache_read": 0.2,
|
|
21
|
+
"output": 20.0,
|
|
22
|
+
"batch_input": 2.0,
|
|
23
|
+
"batch_output": 10.0
|
|
24
|
+
},
|
|
25
|
+
"claude-fable-5-2": {
|
|
26
|
+
"input": 8.0,
|
|
27
|
+
"cache_write_5m": 10.0,
|
|
28
|
+
"cache_write_1h": 16.0,
|
|
29
|
+
"cache_read": 0.2,
|
|
30
|
+
"output": 40.0,
|
|
31
|
+
"batch_input": 4.0,
|
|
32
|
+
"batch_output": 20.0
|
|
33
|
+
}
|
|
34
|
+
}
|
|
35
|
+
}
|