leanscreen 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,10 @@
1
+ __pycache__/
2
+ *.pyc
3
+ .venv/
4
+ .env
5
+ dist/
6
+ build/
7
+ *.egg-info/
8
+ .mypy_cache/
9
+ .ruff_cache/
10
+ .pytest_cache/
@@ -0,0 +1,105 @@
1
+ # Functional Source License, Version 1.1, Apache 2.0 Future License
2
+
3
+ ## Abbreviation
4
+
5
+ FSL-1.1-Apache-2.0
6
+
7
+ ## Notice
8
+
9
+ Copyright 2026 Millennium Research
10
+
11
+ ## Terms and Conditions
12
+
13
+ ### Licensor ("We")
14
+
15
+ The party offering the Software under these Terms and Conditions.
16
+
17
+ ### The Software
18
+
19
+ The "Software" is each version of the software that we make available under
20
+ these Terms and Conditions, as indicated by our inclusion of these Terms and
21
+ Conditions with the Software.
22
+
23
+ ### License Grant
24
+
25
+ Subject to your compliance with this License Grant and the Patents,
26
+ Redistribution and Trademark clauses below, we hereby grant you the right to
27
+ use, copy, modify, create derivative works, publicly perform, publicly display
28
+ and redistribute the Software for any Permitted Purpose identified below.
29
+
30
+ ### Permitted Purpose
31
+
32
+ A Permitted Purpose is any purpose other than a Competing Use. A Competing Use
33
+ means making the Software available to others in a commercial product or
34
+ service that:
35
+
36
+ 1. substitutes for the Software;
37
+
38
+ 2. substitutes for any other product or service we offer using the Software
39
+ that exists as of the date we make the Software available; or
40
+
41
+ 3. offers the same or substantially similar functionality as the Software.
42
+
43
+ Permitted Purposes specifically include using the Software:
44
+
45
+ 1. for your internal use and access;
46
+
47
+ 2. for non-commercial education;
48
+
49
+ 3. for non-commercial research; and
50
+
51
+ 4. in connection with professional services that you provide to a licensee
52
+ using the Software in accordance with these Terms and Conditions.
53
+
54
+ ### Patents
55
+
56
+ To the extent your use for a Permitted Purpose would necessarily infringe our
57
+ patents, the license grant above includes a license under our patents. If you
58
+ make a claim against any party that the Software infringes or contributes to
59
+ the infringement of any patent, then your patent license to the Software ends
60
+ immediately.
61
+
62
+ ### Redistribution
63
+
64
+ The Terms and Conditions apply to all copies, modifications and derivatives of
65
+ the Software.
66
+
67
+ If you redistribute any copies, modifications or derivatives of the Software,
68
+ you must include a copy of or a link to these Terms and Conditions and not
69
+ remove any copyright notices provided in or with the Software.
70
+
71
+ ### Disclaimer
72
+
73
+ THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR
74
+ IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR
75
+ PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT.
76
+
77
+ IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE
78
+ SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES,
79
+ EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE.
80
+
81
+ ### Trademarks
82
+
83
+ Except for displaying the License Details and identifying us as the origin of
84
+ the Software, you have no right under these Terms and Conditions to use our
85
+ trademarks, trade names, service marks or product names.
86
+
87
+ ## Grant of Future License
88
+
89
+ We hereby irrevocably grant you an additional license to use the Software under
90
+ the Apache License, Version 2.0 that is effective on the second anniversary of
91
+ the date we make the Software available. On or after that date, you may use the
92
+ Software under the Apache License, Version 2.0, in which case the following
93
+ will apply:
94
+
95
+ Licensed under the Apache License, Version 2.0 (the "License"); you may not use
96
+ this file except in compliance with the License.
97
+
98
+ You may obtain a copy of the License at
99
+
100
+ http://www.apache.org/licenses/LICENSE-2.0
101
+
102
+ Unless required by applicable law or agreed to in writing, software distributed
103
+ under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
104
+ CONDITIONS OF ANY KIND, either express or implied. See the License for the
105
+ specific language governing permissions and limitations under the License.
@@ -0,0 +1,255 @@
1
+ Metadata-Version: 2.4
2
+ Name: leanscreen
3
+ Version: 0.1.0
4
+ Summary: A calibrated faithfulness screen for informal↔Lean 4 statement pairs, served over MCP. Screens only — never certifies.
5
+ Project-URL: Homepage, https://millenniumresearch.ai/leanscreen
6
+ Project-URL: Repository, https://github.com/ibrahimmian36/leanscreen
7
+ Author-email: Millennium Research <ibrahimnmian@gmail.com>
8
+ License: # Functional Source License, Version 1.1, Apache 2.0 Future License
9
+
10
+ ## Abbreviation
11
+
12
+ FSL-1.1-Apache-2.0
13
+
14
+ ## Notice
15
+
16
+ Copyright 2026 Millennium Research
17
+
18
+ ## Terms and Conditions
19
+
20
+ ### Licensor ("We")
21
+
22
+ The party offering the Software under these Terms and Conditions.
23
+
24
+ ### The Software
25
+
26
+ The "Software" is each version of the software that we make available under
27
+ these Terms and Conditions, as indicated by our inclusion of these Terms and
28
+ Conditions with the Software.
29
+
30
+ ### License Grant
31
+
32
+ Subject to your compliance with this License Grant and the Patents,
33
+ Redistribution and Trademark clauses below, we hereby grant you the right to
34
+ use, copy, modify, create derivative works, publicly perform, publicly display
35
+ and redistribute the Software for any Permitted Purpose identified below.
36
+
37
+ ### Permitted Purpose
38
+
39
+ A Permitted Purpose is any purpose other than a Competing Use. A Competing Use
40
+ means making the Software available to others in a commercial product or
41
+ service that:
42
+
43
+ 1. substitutes for the Software;
44
+
45
+ 2. substitutes for any other product or service we offer using the Software
46
+ that exists as of the date we make the Software available; or
47
+
48
+ 3. offers the same or substantially similar functionality as the Software.
49
+
50
+ Permitted Purposes specifically include using the Software:
51
+
52
+ 1. for your internal use and access;
53
+
54
+ 2. for non-commercial education;
55
+
56
+ 3. for non-commercial research; and
57
+
58
+ 4. in connection with professional services that you provide to a licensee
59
+ using the Software in accordance with these Terms and Conditions.
60
+
61
+ ### Patents
62
+
63
+ To the extent your use for a Permitted Purpose would necessarily infringe our
64
+ patents, the license grant above includes a license under our patents. If you
65
+ make a claim against any party that the Software infringes or contributes to
66
+ the infringement of any patent, then your patent license to the Software ends
67
+ immediately.
68
+
69
+ ### Redistribution
70
+
71
+ The Terms and Conditions apply to all copies, modifications and derivatives of
72
+ the Software.
73
+
74
+ If you redistribute any copies, modifications or derivatives of the Software,
75
+ you must include a copy of or a link to these Terms and Conditions and not
76
+ remove any copyright notices provided in or with the Software.
77
+
78
+ ### Disclaimer
79
+
80
+ THE SOFTWARE IS PROVIDED "AS IS" AND WITHOUT WARRANTIES OF ANY KIND, EXPRESS OR
81
+ IMPLIED, INCLUDING WITHOUT LIMITATION WARRANTIES OF FITNESS FOR A PARTICULAR
82
+ PURPOSE, MERCHANTABILITY, TITLE OR NON-INFRINGEMENT.
83
+
84
+ IN NO EVENT WILL WE HAVE ANY LIABILITY TO YOU ARISING OUT OF OR RELATED TO THE
85
+ SOFTWARE, INCLUDING INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES,
86
+ EVEN IF WE HAVE BEEN INFORMED OF THEIR POSSIBILITY IN ADVANCE.
87
+
88
+ ### Trademarks
89
+
90
+ Except for displaying the License Details and identifying us as the origin of
91
+ the Software, you have no right under these Terms and Conditions to use our
92
+ trademarks, trade names, service marks or product names.
93
+
94
+ ## Grant of Future License
95
+
96
+ We hereby irrevocably grant you an additional license to use the Software under
97
+ the Apache License, Version 2.0 that is effective on the second anniversary of
98
+ the date we make the Software available. On or after that date, you may use the
99
+ Software under the Apache License, Version 2.0, in which case the following
100
+ will apply:
101
+
102
+ Licensed under the Apache License, Version 2.0 (the "License"); you may not use
103
+ this file except in compliance with the License.
104
+
105
+ You may obtain a copy of the License at
106
+
107
+ http://www.apache.org/licenses/LICENSE-2.0
108
+
109
+ Unless required by applicable law or agreed to in writing, software distributed
110
+ under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR
111
+ CONDITIONS OF ANY KIND, either express or implied. See the License for the
112
+ specific language governing permissions and limitations under the License.
113
+ License-File: LICENSE
114
+ Keywords: autoformalization,faithfulness,lean4,mathlib,mcp
115
+ Requires-Python: <3.14,>=3.12
116
+ Requires-Dist: httpx>=0.27.0
117
+ Requires-Dist: mcp<2,>=1.28.0
118
+ Requires-Dist: pydantic-settings>=2.3.0
119
+ Requires-Dist: pydantic>=2.7.0
120
+ Provides-Extra: dev
121
+ Requires-Dist: mypy>=1.10.0; extra == 'dev'
122
+ Requires-Dist: pytest>=8.2.0; extra == 'dev'
123
+ Requires-Dist: ruff>=0.5.0; extra == 'dev'
124
+ Description-Content-Type: text/markdown
125
+
126
+ # leanscreen
127
+
128
+ A calibrated faithfulness screen for informal↔Lean 4 statement pairs, served
129
+ over [MCP](https://modelcontextprotocol.io) so Claude (Code, Desktop, or any
130
+ MCP client) can check statements **while you draft them**.
131
+
132
+ **The one thing to understand before using it:** this screen may only
133
+ *reject*. `passed_screening` means "no defect found by this harness" — it is
134
+ **not** a certification of faithfulness. Measured against 886 frozen human
135
+ verdicts, statements a human reviewer had rejected still passed the full
136
+ screen 17.0% of the time for theorems and 35.6% for definitions; statements
137
+ a human had certified faithful were flagged 15–18% of the time. Every
138
+ response carries this calibration verbatim.
139
+
140
+ ## Two tools
141
+
142
+ **`check_fast`** — deterministic only: lints (unused binders, trivially
143
+ satisfiable existentials, pinned `∃!` witnesses, suspicious ℕ-arithmetic,
144
+ …), vacuity checks (reflexive goals, `True` goals, withheld declarations),
145
+ and Lean 4 elaboration against your own mathlib environment. **Zero API
146
+ calls, no key needed, ~0.1s per statement once the REPL is warm.** Call it
147
+ constantly while drafting.
148
+
149
+ **`check_deep`** — everything in `check_fast`, plus two independent LLM
150
+ judges under strict consensus (a back-translation judge and a
151
+ clause-by-clause checklist judge on separate models) and an adversarial
152
+ counterexample probe. Runs on **your** `ANTHROPIC_API_KEY`; measured cost is
153
+ roughly $0.17–0.27 per statement (the response reports actual spend as
154
+ `actual_cost_usd`), 30–60 seconds. Call it deliberately, before something
155
+ ships.
156
+
157
+ Both take `informal` (the natural-language statement), `lean` (the Lean 4
158
+ statement), and an optional `kind` (`theorem` | `definition`, inferred from
159
+ the declaration head when omitted). Responses rank their evidence —
160
+ `counterexample` > `deterministic` > `two-judge-consensus` > `single-judge`
161
+ — and a single-judge flag is explicitly labeled as below the reporting bar.
162
+
163
+ ## Install
164
+
165
+ ```bash
166
+ pip install git+https://github.com/ibrahimmian36/leanscreen
167
+ ```
168
+
169
+ Requires Python ≥3.12. Runtime dependencies: `httpx`, `pydantic`,
170
+ `pydantic-settings`, `mcp` — nothing else.
171
+
172
+ ## Lean setup (optional but recommended)
173
+
174
+ Without a Lean project, the server still runs — `check_fast` does lints +
175
+ vacuity and says plainly that elaboration was skipped. With one, statements
176
+ are elaborated for real:
177
+
178
+ 1. A Lean 4 project with mathlib, built: `lake build` inside it.
179
+ 2. The [community REPL](https://github.com/leanprover-community/repl),
180
+ built against the **same toolchain**: `lake build` inside the repl repo
181
+ gives you `.lake/build/bin/repl`.
182
+ 3. `lake` on the server's PATH.
183
+
184
+ mathlib imports once at server startup (~100 seconds, in the background —
185
+ calls arriving mid-warm-up answer immediately with a "still warming" note),
186
+ then each check takes ~0.1s.
187
+
188
+ ## Configuration
189
+
190
+ Environment variables (or a `.env` in the working directory), all
191
+ `LEANSCREEN_`-prefixed:
192
+
193
+ | Variable | Default | Meaning |
194
+ |---|---|---|
195
+ | `LEANSCREEN_LEAN_PROJECT_PATH` | unset | Lean 4 + mathlib project (elaboration off when unset) |
196
+ | `LEANSCREEN_LEAN_REPL_PATH` | unset | community REPL binary; without it every check pays a full `lake env lean` |
197
+ | `LEANSCREEN_LEAN_TIMEOUT_SECONDS` | `180` | per-statement Lean budget |
198
+ | `LEANSCREEN_ANTHROPIC_MODEL` | `claude-opus-4-8` | judge A + probe (the calibrated default) |
199
+ | `LEANSCREEN_JUDGE_B_MODEL` | `claude-fable-5` | checklist judge (calibrated default; locked-surface models get a 32k token budget automatically) |
200
+ | `LEANSCREEN_MAX_TOKENS` | `4096` | judge A response budget |
201
+ | `ANTHROPIC_API_KEY` | unset | needed for `check_deep` only |
202
+
203
+ Claude Code (`.mcp.json` in your project) or Claude Desktop
204
+ (`claude_desktop_config.json`):
205
+
206
+ ```json
207
+ {
208
+ "mcpServers": {
209
+ "lean-faithfulness-screen": {
210
+ "command": "leanscreen",
211
+ "env": {
212
+ "LEANSCREEN_LEAN_PROJECT_PATH": "/path/to/your/lean-mathlib-project",
213
+ "LEANSCREEN_LEAN_REPL_PATH": "/path/to/repl/.lake/build/bin/repl",
214
+ "ANTHROPIC_API_KEY": "sk-ant-…"
215
+ }
216
+ }
217
+ }
218
+ }
219
+ ```
220
+
221
+ ## What this does not guarantee
222
+
223
+ The judge configuration was calibrated 2026-07-15 against 886 frozen human
224
+ verdicts (595 faithful / 291 unfaithful) from a production research-math
225
+ corpus. Under strict two-judge consensus, human-rejected pairs still passed
226
+ 17.0% (theorems) / 35.6% (definitions) of the time, and human-certified
227
+ pairs were flagged 15–18% of the time. Both judges are Anthropic-family
228
+ models, so correlated blind spots cannot be ruled out. The counterexample
229
+ probe confabulates: on one PutnamBench sample its counterexamples were wrong
230
+ 4 times out of 5. **Treat every flag as a candidate for human confirmation
231
+ and every pass as "nothing found", never "faithful."**
232
+
233
+ Human certification — an expert reviewer confirming that the Lean means the
234
+ informal statement — is what this screen deliberately does not automate. We
235
+ offer it as a service: **contact ibrahimnmian@gmail.com**.
236
+
237
+ ## License
238
+
239
+ [FSL-1.1-Apache-2.0](LICENSE) (the Functional Source License): free to
240
+ use, copy, modify, and redistribute — including internal commercial use,
241
+ non-commercial education and research, and professional services — but not
242
+ to offer as a competing commercial product or service. **Each version
243
+ automatically becomes Apache 2.0 two years after its release** (the same
244
+ license as mathlib). Not OSI-approved until the conversion — read it
245
+ before building on it commercially.
246
+
247
+ ## Provenance
248
+
249
+ Extracted from Millennium Research's private formalization platform
250
+ (2026-07-28); the detector stack, judge prompts, and calibration figures are
251
+ the ones behind our benchmark audits — the miniF2F and ProofNet# filings are
252
+ public, and the PutnamBench, ProofNetVerif, and CLEVER audits have been
253
+ shared with their maintainers. The calibration *data* is not included.
254
+
255
+ Project page: [millenniumresearch.ai/leanscreen](https://millenniumresearch.ai/leanscreen)
@@ -0,0 +1,130 @@
1
+ # leanscreen
2
+
3
+ A calibrated faithfulness screen for informal↔Lean 4 statement pairs, served
4
+ over [MCP](https://modelcontextprotocol.io) so Claude (Code, Desktop, or any
5
+ MCP client) can check statements **while you draft them**.
6
+
7
+ **The one thing to understand before using it:** this screen may only
8
+ *reject*. `passed_screening` means "no defect found by this harness" — it is
9
+ **not** a certification of faithfulness. Measured against 886 frozen human
10
+ verdicts, statements a human reviewer had rejected still passed the full
11
+ screen 17.0% of the time for theorems and 35.6% for definitions; statements
12
+ a human had certified faithful were flagged 15–18% of the time. Every
13
+ response carries this calibration verbatim.
14
+
15
+ ## Two tools
16
+
17
+ **`check_fast`** — deterministic only: lints (unused binders, trivially
18
+ satisfiable existentials, pinned `∃!` witnesses, suspicious ℕ-arithmetic,
19
+ …), vacuity checks (reflexive goals, `True` goals, withheld declarations),
20
+ and Lean 4 elaboration against your own mathlib environment. **Zero API
21
+ calls, no key needed, ~0.1s per statement once the REPL is warm.** Call it
22
+ constantly while drafting.
23
+
24
+ **`check_deep`** — everything in `check_fast`, plus two independent LLM
25
+ judges under strict consensus (a back-translation judge and a
26
+ clause-by-clause checklist judge on separate models) and an adversarial
27
+ counterexample probe. Runs on **your** `ANTHROPIC_API_KEY`; measured cost is
28
+ roughly $0.17–0.27 per statement (the response reports actual spend as
29
+ `actual_cost_usd`), 30–60 seconds. Call it deliberately, before something
30
+ ships.
31
+
32
+ Both take `informal` (the natural-language statement), `lean` (the Lean 4
33
+ statement), and an optional `kind` (`theorem` | `definition`, inferred from
34
+ the declaration head when omitted). Responses rank their evidence —
35
+ `counterexample` > `deterministic` > `two-judge-consensus` > `single-judge`
36
+ — and a single-judge flag is explicitly labeled as below the reporting bar.
37
+
38
+ ## Install
39
+
40
+ ```bash
41
+ pip install git+https://github.com/ibrahimmian36/leanscreen
42
+ ```
43
+
44
+ Requires Python ≥3.12. Runtime dependencies: `httpx`, `pydantic`,
45
+ `pydantic-settings`, `mcp` — nothing else.
46
+
47
+ ## Lean setup (optional but recommended)
48
+
49
+ Without a Lean project, the server still runs — `check_fast` does lints +
50
+ vacuity and says plainly that elaboration was skipped. With one, statements
51
+ are elaborated for real:
52
+
53
+ 1. A Lean 4 project with mathlib, built: `lake build` inside it.
54
+ 2. The [community REPL](https://github.com/leanprover-community/repl),
55
+ built against the **same toolchain**: `lake build` inside the repl repo
56
+ gives you `.lake/build/bin/repl`.
57
+ 3. `lake` on the server's PATH.
58
+
59
+ mathlib imports once at server startup (~100 seconds, in the background —
60
+ calls arriving mid-warm-up answer immediately with a "still warming" note),
61
+ then each check takes ~0.1s.
62
+
63
+ ## Configuration
64
+
65
+ Environment variables (or a `.env` in the working directory), all
66
+ `LEANSCREEN_`-prefixed:
67
+
68
+ | Variable | Default | Meaning |
69
+ |---|---|---|
70
+ | `LEANSCREEN_LEAN_PROJECT_PATH` | unset | Lean 4 + mathlib project (elaboration off when unset) |
71
+ | `LEANSCREEN_LEAN_REPL_PATH` | unset | community REPL binary; without it every check pays a full `lake env lean` |
72
+ | `LEANSCREEN_LEAN_TIMEOUT_SECONDS` | `180` | per-statement Lean budget |
73
+ | `LEANSCREEN_ANTHROPIC_MODEL` | `claude-opus-4-8` | judge A + probe (the calibrated default) |
74
+ | `LEANSCREEN_JUDGE_B_MODEL` | `claude-fable-5` | checklist judge (calibrated default; locked-surface models get a 32k token budget automatically) |
75
+ | `LEANSCREEN_MAX_TOKENS` | `4096` | judge A response budget |
76
+ | `ANTHROPIC_API_KEY` | unset | needed for `check_deep` only |
77
+
78
+ Claude Code (`.mcp.json` in your project) or Claude Desktop
79
+ (`claude_desktop_config.json`):
80
+
81
+ ```json
82
+ {
83
+ "mcpServers": {
84
+ "lean-faithfulness-screen": {
85
+ "command": "leanscreen",
86
+ "env": {
87
+ "LEANSCREEN_LEAN_PROJECT_PATH": "/path/to/your/lean-mathlib-project",
88
+ "LEANSCREEN_LEAN_REPL_PATH": "/path/to/repl/.lake/build/bin/repl",
89
+ "ANTHROPIC_API_KEY": "sk-ant-…"
90
+ }
91
+ }
92
+ }
93
+ }
94
+ ```
95
+
96
+ ## What this does not guarantee
97
+
98
+ The judge configuration was calibrated 2026-07-15 against 886 frozen human
99
+ verdicts (595 faithful / 291 unfaithful) from a production research-math
100
+ corpus. Under strict two-judge consensus, human-rejected pairs still passed
101
+ 17.0% (theorems) / 35.6% (definitions) of the time, and human-certified
102
+ pairs were flagged 15–18% of the time. Both judges are Anthropic-family
103
+ models, so correlated blind spots cannot be ruled out. The counterexample
104
+ probe confabulates: on one PutnamBench sample its counterexamples were wrong
105
+ 4 times out of 5. **Treat every flag as a candidate for human confirmation
106
+ and every pass as "nothing found", never "faithful."**
107
+
108
+ Human certification — an expert reviewer confirming that the Lean means the
109
+ informal statement — is what this screen deliberately does not automate. We
110
+ offer it as a service: **contact ibrahimnmian@gmail.com**.
111
+
112
+ ## License
113
+
114
+ [FSL-1.1-Apache-2.0](LICENSE) (the Functional Source License): free to
115
+ use, copy, modify, and redistribute — including internal commercial use,
116
+ non-commercial education and research, and professional services — but not
117
+ to offer as a competing commercial product or service. **Each version
118
+ automatically becomes Apache 2.0 two years after its release** (the same
119
+ license as mathlib). Not OSI-approved until the conversion — read it
120
+ before building on it commercially.
121
+
122
+ ## Provenance
123
+
124
+ Extracted from Millennium Research's private formalization platform
125
+ (2026-07-28); the detector stack, judge prompts, and calibration figures are
126
+ the ones behind our benchmark audits — the miniF2F and ProofNet# filings are
127
+ public, and the PutnamBench, ProofNetVerif, and CLEVER audits have been
128
+ shared with their maintainers. The calibration *data* is not included.
129
+
130
+ Project page: [millenniumresearch.ai/leanscreen](https://millenniumresearch.ai/leanscreen)
@@ -0,0 +1,112 @@
1
+ # Build prompt — extract the screen into `leanscreen`, ready for public release
2
+
3
+ > This is the brief the extraction was executed against (2026-07-28). It is
4
+ > written so a fresh Claude Code session could run it end to end; it also
5
+ > records every decision made, so future maintainers know what was deliberate.
6
+
7
+ ---
8
+
9
+ I want the MCP faithfulness screener extracted from the private
10
+ `arxiv-math-platform` repo into a standalone, publicly publishable package.
11
+ The private repo stays the product; the public package is the funnel.
12
+
13
+ ## What the product is
14
+
15
+ `leanscreen`: one pip-installable package, one console script (`leanscreen`),
16
+ serving the two MCP tools that already exist — `check_fast` (free,
17
+ deterministic: lints + vacuity + Lean elaboration) and `check_deep` (paid:
18
+ dual-judge strict consensus + counterexample probe, on the user's own
19
+ `ANTHROPIC_API_KEY`). Local-first, single-user, no hosting, no auth, no
20
+ telemetry.
21
+
22
+ ## What moves, what stays private
23
+
24
+ **Moves** (the screen subtree — it depends only on `httpx` at runtime):
25
+ `llm.py`, `backtranslate.py` (judge prompts), `lints.py`, `quality.py`,
26
+ `falsify.py`, `verify.py`, `lean_server.py`, `logging.py`, the single-pair
27
+ `score_pair` core of `score_external.py`, and the MCP server. Unit tests for
28
+ each module come along.
29
+
30
+ **Stays private** (the commercial harness and everything upstream of it):
31
+ the batch layer (`load_pairs` / `run` / `render_report` / budget rails /
32
+ crash-safe resume), the numeric cross-evaluation probe and its sandbox, the
33
+ whole data platform (ingestion, extraction, DB, review UI, export,
34
+ benchmark, grader), the 886-verdict calibration set, and every census
35
+ artifact. The public package quotes the calibration *numbers*; the data that
36
+ produced them is the moat and never ships.
37
+
38
+ ## Decisions (made, revisit deliberately)
39
+
40
+ - **Name**: `leanscreen`; package `leanscreen`, script `leanscreen`, MCP
41
+ server name stays `lean-faithfulness-screen`.
42
+ - **License**: FSL-1.1-Apache-2.0 (revised from ELv2 on 2026-07-28 after
43
+ license research, on Ibby's request). Why FSL won: (1) its Competing Use
44
+ clause covers "any other product or service we offer using the Software" —
45
+ protecting the certification/screening *service*, where ELv2 only blocks
46
+ hosted versions of the software itself; (2) Permitted Purposes explicitly
47
+ allow internal use, non-commercial education, and research — the mathlib
48
+ audience; (3) each version converts to Apache 2.0 (mathlib's license)
49
+ after 2 years, the best community optics available, and a 2-year-old
50
+ screener without current calibration is not a competitive threat; (4) it
51
+ is simpler than BUSL, whose Additional Use Grant ambiguity is exactly what
52
+ Sentry created FSL to fix. (Not legal advice; have a lawyer confirm before
53
+ anything heavy rides on it.)
54
+ - **Env prefix**: `LEANSCREEN_` (`LEAN_PROJECT_PATH`, `LEAN_REPL_PATH`,
55
+ `ANTHROPIC_MODEL`, `JUDGE_B_MODEL`, `MAX_TOKENS`, …). A fresh, slim
56
+ `Settings` — none of the platform's 50 knobs.
57
+ - **Judge B is now a setting** (`LEANSCREEN_JUDGE_B_MODEL`, default
58
+ `claude-fable-5` = the calibrated configuration) instead of a hard-coded
59
+ constant; the locked-surface 32k max_tokens rule keys off the value.
60
+ - **Numeric probe dropped** from the public `score_pair` (the MCP tools never
61
+ used it; smaller surface, smaller audit).
62
+ - **`infer_kind` made public** (was `_infer_kind`).
63
+ - **Fork policy**: this is a fork-with-provenance, not a shared library. The
64
+ private repo does NOT depend on the public package. When a detector
65
+ improves in one repo, port it to the other deliberately; the commit
66
+ message must name the source commit.
67
+
68
+ ## Non-negotiable constraints (carried over verbatim)
69
+
70
+ - The screen may only reject — never certify. `CALIBRATION_DISCLOSURE` ships
71
+ verbatim in every payload; both tool descriptions state passing is not
72
+ certification (test-pinned).
73
+ - Evidence tiers stay ranked, never flattened: counterexample >
74
+ deterministic > two-judge-consensus > single-judge (below the reporting
75
+ bar).
76
+ - Every flag is a candidate, not a verdict.
77
+ - No secrets, no internal paths, no customer or reviewer names anywhere in
78
+ the public tree — audit before every push.
79
+
80
+ ## Quality gate
81
+
82
+ ```bash
83
+ .venv/bin/ruff check . && .venv/bin/ruff format --check . && .venv/bin/mypy && .venv/bin/pytest
84
+ ```
85
+
86
+ Strict mypy, zero errors, same bar as the private repo.
87
+
88
+ ## Acceptance criteria
89
+
90
+ 1. `pip install -e .` in a fresh venv pulls only httpx, pydantic(-settings),
91
+ and mcp (plus their transitive deps) — none of the platform's heavy deps.
92
+ 2. The full test suite passes; the MCP-server tests are the same ones that
93
+ pass in the private repo, re-pointed.
94
+ 3. `leanscreen` starts over stdio with no Lean project and no API key and
95
+ answers `check_fast`, exactly as the private server does.
96
+ 4. README works for a stranger: REPL build walkthrough, generic paths,
97
+ config snippets for Claude Code / Claude Desktop, the calibration table,
98
+ the ELv2 summary, and a contact for certification engagements.
99
+ 5. A pre-publish audit pass over the whole tree: secret scan, internal-path
100
+ scan, license header presence, README claims match measured numbers.
101
+
102
+ ## Steps reserved for Ibby (do not automate)
103
+
104
+ 1. Confirm the license choice and the public contact address.
105
+ 2. `gh repo create` (public) + first push — publishing is a send.
106
+ 3. Any PyPI upload, registry listing, or announcement post (10am rule).
107
+
108
+ ---
109
+
110
+ *Executed 2026-07-28. Origin: arxiv-math-platform @ 0f57167 (commits e39d2f9
111
+ + 0f57167 are the MCP server and the proof-stub fix this package forked
112
+ from).*