econworkbench 1.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- econworkbench-1.2.0/LICENSE +21 -0
- econworkbench-1.2.0/PKG-INFO +150 -0
- econworkbench-1.2.0/README.md +111 -0
- econworkbench-1.2.0/pyproject.toml +28 -0
- econworkbench-1.2.0/setup.cfg +4 -0
- econworkbench-1.2.0/src/econworkbench/__init__.py +1 -0
- econworkbench-1.2.0/src/econworkbench/crosscheck.py +111 -0
- econworkbench-1.2.0/src/econworkbench/report.py +198 -0
- econworkbench-1.2.0/src/econworkbench.egg-info/PKG-INFO +150 -0
- econworkbench-1.2.0/src/econworkbench.egg-info/SOURCES.txt +12 -0
- econworkbench-1.2.0/src/econworkbench.egg-info/dependency_links.txt +1 -0
- econworkbench-1.2.0/src/econworkbench.egg-info/entry_points.txt +3 -0
- econworkbench-1.2.0/src/econworkbench.egg-info/requires.txt +4 -0
- econworkbench-1.2.0/src/econworkbench.egg-info/top_level.txt +1 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 ljftwq-dev
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: econworkbench
|
|
3
|
+
Version: 1.2.0
|
|
4
|
+
Summary: From idea to referee-ready: an open workbench for empirical research. Core: crosscheck (R/Python/Stata triple validation) + report (journal-ready three-line tables).
|
|
5
|
+
Author: ljftwq-dev
|
|
6
|
+
License: MIT License
|
|
7
|
+
|
|
8
|
+
Copyright (c) 2026 ljftwq-dev
|
|
9
|
+
|
|
10
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
11
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
12
|
+
in the Software without restriction, including without limitation the rights
|
|
13
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
14
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
15
|
+
furnished to do so, subject to the following conditions:
|
|
16
|
+
|
|
17
|
+
The above copyright notice and this permission notice shall be included in all
|
|
18
|
+
copies or substantial portions of the Software.
|
|
19
|
+
|
|
20
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
21
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
22
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
23
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
24
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
25
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
26
|
+
SOFTWARE.
|
|
27
|
+
|
|
28
|
+
Keywords: econometrics,causal-inference,difference-in-differences,stata,r,reproducibility
|
|
29
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
30
|
+
Classifier: Programming Language :: Python :: 3
|
|
31
|
+
Classifier: Topic :: Scientific/Engineering
|
|
32
|
+
Requires-Python: >=3.9
|
|
33
|
+
Description-Content-Type: text/markdown
|
|
34
|
+
License-File: LICENSE
|
|
35
|
+
Provides-Extra: report
|
|
36
|
+
Requires-Dist: python-docx; extra == "report"
|
|
37
|
+
Requires-Dist: matplotlib; extra == "report"
|
|
38
|
+
Dynamic: license-file
|
|
39
|
+
|
|
40
|
+
# EconWorkbench
|
|
41
|
+
|
|
42
|
+
[](https://github.com/ljftwq-dev/EconWorkbench/actions/workflows/ci.yml)
|
|
43
|
+
[](LICENSE)
|
|
44
|
+
|
|
45
|
+
[English](README.md) | [中文](README_CN.md)
|
|
46
|
+
|
|
47
|
+
**From idea to referee-ready — an open workbench for empirical research.**
|
|
48
|
+
|
|
49
|
+
EconWorkbench is a toolkit for the empirical-research workflow: literature, data, design, estimation, **verification**, and reporting. Research judgment stays with you — the workbench makes each step faster and the numbers trustworthy.
|
|
50
|
+
|
|
51
|
+
**v1.0 ships the core: `crosscheck/`** — run the *same* model in **R, Python, and Stata**, then diff the results automatically. If the numbers agree, your code is (almost certainly) right. If they don't, you stop interpreting — before a reviewer (or a referee report) stops you.
|
|
52
|
+
|
|
53
|
+
<p align="center">
|
|
54
|
+
<img src="assets/triple_crosscheck_cs2021.png" alt="R, Python and Stata terminals side by side reproducing Callaway-Sant'Anna (2021), all showing ATT = -0.0399513" width="900">
|
|
55
|
+
</p>
|
|
56
|
+
|
|
57
|
+
*Live screenshot: Callaway & Sant'Anna (2021) staggered DiD on `mpdta`, run independently in R (`did`), Python (`StatsPAI`), and Stata (`csdid`). All three report Overall ATT = **-0.0399513**.*
|
|
58
|
+
|
|
59
|
+
## Why
|
|
60
|
+
|
|
61
|
+
AI agents write most of our econometrics code now. But an LLM that writes the wrong `vcov` in two languages — consistently — will hand you confident nonsense. The fix is older than LLMs: **independent implementations of the same estimator should agree**.
|
|
62
|
+
|
|
63
|
+
- R's `did` (Callaway et al.), Stata's `csdid` (Sant'Anna et al.), and Python ports are written by different teams. Same data + same model + same numbers = implementation is clean.
|
|
64
|
+
- One run, one number, no cross-check = you cannot tell a result from a bug.
|
|
65
|
+
|
|
66
|
+
This is the "double-entry bookkeeping" for regressions — inspired by a 2026 lecture series on AI agents for social-science research (where the speaker, priced out of Stata licenses, could only cross-check R vs Python), and by the [StatsPAI](https://github.com/brycewang-stanford/StatsPAI) parity index, which grades its own estimators against R/Stata references.
|
|
67
|
+
|
|
68
|
+
## Quick start
|
|
69
|
+
|
|
70
|
+
Core is zero-dependency (pure stdlib):
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
pip install econworkbench # or: pip install .[report] for tables
|
|
74
|
+
|
|
75
|
+
# 1. Run the same model in R / Python / Stata; each script writes a 4-column CSV:
|
|
76
|
+
# term, estimate, se, pvalue
|
|
77
|
+
Rscript my_model.R # -> r_results.csv
|
|
78
|
+
python my_model.py # -> py_results.csv
|
|
79
|
+
"C:\Program Files\Stata19\StataSE-64.exe" /e do my_model.do # -> stata_results.csv
|
|
80
|
+
|
|
81
|
+
# 2. Diff them (first CSV is the reference):
|
|
82
|
+
econ-crosscheck r_results.csv py_results.csv stata_results.csv
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Output:
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
term r_resultsvspy_resultsr_resultsvsstata_results
|
|
89
|
+
--------------------------------------------------------------------
|
|
90
|
+
g2004_t2004 bit-exact aligned
|
|
91
|
+
...
|
|
92
|
+
overall_simple bit-exact aligned
|
|
93
|
+
--------------------------------------------------------------------
|
|
94
|
+
r_results vs py_results: bit-exact 8 | aligned 0 | FAIL 0
|
|
95
|
+
r_results vs stata_results: bit-exact 0 | aligned 8 | FAIL 0
|
|
96
|
+
RESULT: PASS (aligned) — exit code 0
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
## Verdicts
|
|
100
|
+
|
|
101
|
+
| Verdict | Meaning | Default tolerance |
|
|
102
|
+
|---|---|---|
|
|
103
|
+
| `bit-exact` | Machine-precision agreement | Δ ≤ 1e-10 |
|
|
104
|
+
| `aligned` | Agreement within tolerance | Δcoef ≤ 1e-6, ΔSE/SE ≤ 1e-4, Δp ≤ 1e-4 |
|
|
105
|
+
| `FAIL` | Significance flips or tolerance breached | — exit code 1 |
|
|
106
|
+
|
|
107
|
+
Exit codes make it gate-able: wire it into CI, a Makefile, or your SDD pipeline as a release gate.
|
|
108
|
+
|
|
109
|
+
## Example: Callaway & Sant'Anna (2021) replication
|
|
110
|
+
|
|
111
|
+
[`examples/cs2021_mpdta/`](examples/cs2021_mpdta/) replicates the canonical staggered-DiD application (minimum-wage effects on teen employment, 2,500 county-year obs) three ways:
|
|
112
|
+
|
|
113
|
+
| | Implementation | Authors' own package |
|
|
114
|
+
|---|---|---|
|
|
115
|
+
| R 4.6.1 | `did::att_gt()` + `aggte()` | Callaway & Sant'Anna |
|
|
116
|
+
| Python 3.14 | `StatsPAI.callaway_santanna()` | port with its own parity evidence |
|
|
117
|
+
| StataNow 19.5 | `csdid` + `csdid_estat simple` | Sant'Anna, Goodman-Bacon & Pedro |
|
|
118
|
+
|
|
119
|
+
Result: **R ↔ Python bit-exact on all 8 quantities** (7 post-period ATT(g,t) + overall); **R ↔ Stata aligned** (Δ ≈ 1e-7, different DR-IPW optimizers). Overall ATT = -0.0399 matches the published paper.
|
|
120
|
+
|
|
121
|
+
Run it yourself: each folder contains the three scripts, the shared dataset (`mpdta_data.csv` — one source of truth, all three languages read the same file), and the three result CSVs.
|
|
122
|
+
|
|
123
|
+
## From estimates to tables (`report/`, v1.2)
|
|
124
|
+
|
|
125
|
+
The same CSVs that feed `crosscheck` also feed `report` — one command from estimates to a journal-ready three-line table:
|
|
126
|
+
|
|
127
|
+
```
|
|
128
|
+
econ-report r_results.csv py_results.csv stata_results.csv --title "Table 1" --out table1
|
|
129
|
+
# -> table1.tex (booktabs) + table1.docx (Word three-line) + table1.png (preview)
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
<p align="center">
|
|
133
|
+
<img src="examples/cs2021_mpdta/table1.png" alt="Three-line regression table comparing R/Python/Stata estimates" width="640">
|
|
134
|
+
</p>
|
|
135
|
+
|
|
136
|
+
Significance stars, SEs in parentheses, top/mid/bottom rules — in all three formats. Column labels sit at the bottom, `esttab`-style.
|
|
137
|
+
|
|
138
|
+
## What it does *not* do
|
|
139
|
+
|
|
140
|
+
- It cannot validate your **identification strategy** — two implementations of a wrong model agree perfectly. (The companion `REVIEWER_CHECKLIST.md` in the parent project covers clustering levels, staggered-DiD pitfalls, pre-trends, etc.)
|
|
141
|
+
- It is not a paper factory. Judgment stays with the researcher.
|
|
142
|
+
|
|
143
|
+
## Related work
|
|
144
|
+
|
|
145
|
+
- [StatsPAI](https://github.com/brycewang-stanford/StatsPAI) — agent-native Stata/R replacement with a queryable parity index (a superset of this idea, inside one library).
|
|
146
|
+
- Sakana AI-Scientist, Agent Laboratory — end-to-end "AI scientist" pipelines (the opposite bet: automation over verification).
|
|
147
|
+
|
|
148
|
+
## License
|
|
149
|
+
|
|
150
|
+
MIT
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# EconWorkbench
|
|
2
|
+
|
|
3
|
+
[](https://github.com/ljftwq-dev/EconWorkbench/actions/workflows/ci.yml)
|
|
4
|
+
[](LICENSE)
|
|
5
|
+
|
|
6
|
+
[English](README.md) | [中文](README_CN.md)
|
|
7
|
+
|
|
8
|
+
**From idea to referee-ready — an open workbench for empirical research.**
|
|
9
|
+
|
|
10
|
+
EconWorkbench is a toolkit for the empirical-research workflow: literature, data, design, estimation, **verification**, and reporting. Research judgment stays with you — the workbench makes each step faster and the numbers trustworthy.
|
|
11
|
+
|
|
12
|
+
**v1.0 ships the core: `crosscheck/`** — run the *same* model in **R, Python, and Stata**, then diff the results automatically. If the numbers agree, your code is (almost certainly) right. If they don't, you stop interpreting — before a reviewer (or a referee report) stops you.
|
|
13
|
+
|
|
14
|
+
<p align="center">
|
|
15
|
+
<img src="assets/triple_crosscheck_cs2021.png" alt="R, Python and Stata terminals side by side reproducing Callaway-Sant'Anna (2021), all showing ATT = -0.0399513" width="900">
|
|
16
|
+
</p>
|
|
17
|
+
|
|
18
|
+
*Live screenshot: Callaway & Sant'Anna (2021) staggered DiD on `mpdta`, run independently in R (`did`), Python (`StatsPAI`), and Stata (`csdid`). All three report Overall ATT = **-0.0399513**.*
|
|
19
|
+
|
|
20
|
+
## Why
|
|
21
|
+
|
|
22
|
+
AI agents write most of our econometrics code now. But an LLM that writes the wrong `vcov` in two languages — consistently — will hand you confident nonsense. The fix is older than LLMs: **independent implementations of the same estimator should agree**.
|
|
23
|
+
|
|
24
|
+
- R's `did` (Callaway et al.), Stata's `csdid` (Sant'Anna et al.), and Python ports are written by different teams. Same data + same model + same numbers = implementation is clean.
|
|
25
|
+
- One run, one number, no cross-check = you cannot tell a result from a bug.
|
|
26
|
+
|
|
27
|
+
This is the "double-entry bookkeeping" for regressions — inspired by a 2026 lecture series on AI agents for social-science research (where the speaker, priced out of Stata licenses, could only cross-check R vs Python), and by the [StatsPAI](https://github.com/brycewang-stanford/StatsPAI) parity index, which grades its own estimators against R/Stata references.
|
|
28
|
+
|
|
29
|
+
## Quick start
|
|
30
|
+
|
|
31
|
+
Core is zero-dependency (pure stdlib):
|
|
32
|
+
|
|
33
|
+
```
|
|
34
|
+
pip install econworkbench # or: pip install .[report] for tables
|
|
35
|
+
|
|
36
|
+
# 1. Run the same model in R / Python / Stata; each script writes a 4-column CSV:
|
|
37
|
+
# term, estimate, se, pvalue
|
|
38
|
+
Rscript my_model.R # -> r_results.csv
|
|
39
|
+
python my_model.py # -> py_results.csv
|
|
40
|
+
"C:\Program Files\Stata19\StataSE-64.exe" /e do my_model.do # -> stata_results.csv
|
|
41
|
+
|
|
42
|
+
# 2. Diff them (first CSV is the reference):
|
|
43
|
+
econ-crosscheck r_results.csv py_results.csv stata_results.csv
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
Output:
|
|
47
|
+
|
|
48
|
+
```
|
|
49
|
+
term r_resultsvspy_resultsr_resultsvsstata_results
|
|
50
|
+
--------------------------------------------------------------------
|
|
51
|
+
g2004_t2004 bit-exact aligned
|
|
52
|
+
...
|
|
53
|
+
overall_simple bit-exact aligned
|
|
54
|
+
--------------------------------------------------------------------
|
|
55
|
+
r_results vs py_results: bit-exact 8 | aligned 0 | FAIL 0
|
|
56
|
+
r_results vs stata_results: bit-exact 0 | aligned 8 | FAIL 0
|
|
57
|
+
RESULT: PASS (aligned) — exit code 0
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
## Verdicts
|
|
61
|
+
|
|
62
|
+
| Verdict | Meaning | Default tolerance |
|
|
63
|
+
|---|---|---|
|
|
64
|
+
| `bit-exact` | Machine-precision agreement | Δ ≤ 1e-10 |
|
|
65
|
+
| `aligned` | Agreement within tolerance | Δcoef ≤ 1e-6, ΔSE/SE ≤ 1e-4, Δp ≤ 1e-4 |
|
|
66
|
+
| `FAIL` | Significance flips or tolerance breached | — exit code 1 |
|
|
67
|
+
|
|
68
|
+
Exit codes make it gate-able: wire it into CI, a Makefile, or your SDD pipeline as a release gate.
|
|
69
|
+
|
|
70
|
+
## Example: Callaway & Sant'Anna (2021) replication
|
|
71
|
+
|
|
72
|
+
[`examples/cs2021_mpdta/`](examples/cs2021_mpdta/) replicates the canonical staggered-DiD application (minimum-wage effects on teen employment, 2,500 county-year obs) three ways:
|
|
73
|
+
|
|
74
|
+
| | Implementation | Authors' own package |
|
|
75
|
+
|---|---|---|
|
|
76
|
+
| R 4.6.1 | `did::att_gt()` + `aggte()` | Callaway & Sant'Anna |
|
|
77
|
+
| Python 3.14 | `StatsPAI.callaway_santanna()` | port with its own parity evidence |
|
|
78
|
+
| StataNow 19.5 | `csdid` + `csdid_estat simple` | Sant'Anna, Goodman-Bacon & Pedro |
|
|
79
|
+
|
|
80
|
+
Result: **R ↔ Python bit-exact on all 8 quantities** (7 post-period ATT(g,t) + overall); **R ↔ Stata aligned** (Δ ≈ 1e-7, different DR-IPW optimizers). Overall ATT = -0.0399 matches the published paper.
|
|
81
|
+
|
|
82
|
+
Run it yourself: each folder contains the three scripts, the shared dataset (`mpdta_data.csv` — one source of truth, all three languages read the same file), and the three result CSVs.
|
|
83
|
+
|
|
84
|
+
## From estimates to tables (`report/`, v1.2)
|
|
85
|
+
|
|
86
|
+
The same CSVs that feed `crosscheck` also feed `report` — one command from estimates to a journal-ready three-line table:
|
|
87
|
+
|
|
88
|
+
```
|
|
89
|
+
econ-report r_results.csv py_results.csv stata_results.csv --title "Table 1" --out table1
|
|
90
|
+
# -> table1.tex (booktabs) + table1.docx (Word three-line) + table1.png (preview)
|
|
91
|
+
```
|
|
92
|
+
|
|
93
|
+
<p align="center">
|
|
94
|
+
<img src="examples/cs2021_mpdta/table1.png" alt="Three-line regression table comparing R/Python/Stata estimates" width="640">
|
|
95
|
+
</p>
|
|
96
|
+
|
|
97
|
+
Significance stars, SEs in parentheses, top/mid/bottom rules — in all three formats. Column labels sit at the bottom, `esttab`-style.
|
|
98
|
+
|
|
99
|
+
## What it does *not* do
|
|
100
|
+
|
|
101
|
+
- It cannot validate your **identification strategy** — two implementations of a wrong model agree perfectly. (The companion `REVIEWER_CHECKLIST.md` in the parent project covers clustering levels, staggered-DiD pitfalls, pre-trends, etc.)
|
|
102
|
+
- It is not a paper factory. Judgment stays with the researcher.
|
|
103
|
+
|
|
104
|
+
## Related work
|
|
105
|
+
|
|
106
|
+
- [StatsPAI](https://github.com/brycewang-stanford/StatsPAI) — agent-native Stata/R replacement with a queryable parity index (a superset of this idea, inside one library).
|
|
107
|
+
- Sakana AI-Scientist, Agent Laboratory — end-to-end "AI scientist" pipelines (the opposite bet: automation over verification).
|
|
108
|
+
|
|
109
|
+
## License
|
|
110
|
+
|
|
111
|
+
MIT
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=61"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "econworkbench"
|
|
7
|
+
version = "1.2.0"
|
|
8
|
+
description = "From idea to referee-ready: an open workbench for empirical research. Core: crosscheck (R/Python/Stata triple validation) + report (journal-ready three-line tables)."
|
|
9
|
+
readme = "README.md"
|
|
10
|
+
license = { file = "LICENSE" }
|
|
11
|
+
authors = [{ name = "ljftwq-dev" }]
|
|
12
|
+
requires-python = ">=3.9"
|
|
13
|
+
keywords = ["econometrics", "causal-inference", "difference-in-differences", "stata", "r", "reproducibility"]
|
|
14
|
+
classifiers = [
|
|
15
|
+
"License :: OSI Approved :: MIT License",
|
|
16
|
+
"Programming Language :: Python :: 3",
|
|
17
|
+
"Topic :: Scientific/Engineering",
|
|
18
|
+
]
|
|
19
|
+
|
|
20
|
+
[project.optional-dependencies]
|
|
21
|
+
report = ["python-docx", "matplotlib"]
|
|
22
|
+
|
|
23
|
+
[project.scripts]
|
|
24
|
+
econ-crosscheck = "econworkbench.crosscheck:main"
|
|
25
|
+
econ-report = "econworkbench.report:main"
|
|
26
|
+
|
|
27
|
+
[tool.setuptools.packages.find]
|
|
28
|
+
where = ["src"]
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
__version__ = "1.2.0"
|
|
@@ -0,0 +1,111 @@
|
|
|
1
|
+
# -*- coding: utf-8 -*-
|
|
2
|
+
"""
|
|
3
|
+
crosscheck.py — EconWorkbench core: cross-language validation engine
|
|
4
|
+
One estimator, independent implementations, one verdict.
|
|
5
|
+
Supports 2~N implementations (R / Python / Stata / ...); first CSV is the reference.
|
|
6
|
+
Grades (StatsPAI-style): bit-exact (<=1e-10) / aligned (tolerance) / FAIL.
|
|
7
|
+
Exit code 0 = PASS, 1 = FAIL (gate-able in CI / Makefile / SDD pipelines).
|
|
8
|
+
"""
|
|
9
|
+
import sys, csv, argparse, os
|
|
10
|
+
|
|
11
|
+
|
|
12
|
+
def build_parser():
|
|
13
|
+
ap = argparse.ArgumentParser(
|
|
14
|
+
prog="econ-crosscheck",
|
|
15
|
+
description="Cross-validate 2~N result CSVs (columns: term,estimate,se,pvalue). First CSV is the reference.")
|
|
16
|
+
ap.add_argument("csvs", nargs="+", help="2~N result CSVs, first is the reference")
|
|
17
|
+
ap.add_argument("--tol-coef", type=float, default=1e-6)
|
|
18
|
+
ap.add_argument("--tol-se", type=float, default=1e-4)
|
|
19
|
+
ap.add_argument("--tol-p", type=float, default=1e-4)
|
|
20
|
+
ap.add_argument("--tol-exact", type=float, default=1e-10, help="bit-exact threshold (machine precision)")
|
|
21
|
+
return ap
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
GRADES = {"bit-exact": 0, "aligned": 1, "FAIL": 2}
|
|
25
|
+
|
|
26
|
+
|
|
27
|
+
def load(path):
|
|
28
|
+
rows = {}
|
|
29
|
+
with open(path, newline="", encoding="utf-8-sig") as f:
|
|
30
|
+
for r in csv.DictReader(f):
|
|
31
|
+
t = r["term"].strip()
|
|
32
|
+
rows[t] = (float(r["estimate"]), float(r["se"]), float(r["pvalue"]))
|
|
33
|
+
return rows
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
def grade(er, sr, pr, ep, sp, pp, args):
|
|
37
|
+
dc = abs(er - ep)
|
|
38
|
+
ds = abs(sr - sp) / max(sr, 1e-12)
|
|
39
|
+
dp = abs(pr - pp)
|
|
40
|
+
if not (pr < 0.05) == (pp < 0.05):
|
|
41
|
+
return "FAIL", dc, ds, dp
|
|
42
|
+
if dc > args.tol_coef or dp > args.tol_p:
|
|
43
|
+
return "FAIL", dc, ds, dp
|
|
44
|
+
if dc <= args.tol_exact and ds <= args.tol_exact and dp <= args.tol_exact:
|
|
45
|
+
return "bit-exact", dc, ds, dp
|
|
46
|
+
if ds > args.tol_se:
|
|
47
|
+
return "FAIL", dc, ds, dp
|
|
48
|
+
return "aligned", dc, ds, dp
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
def main(argv=None):
|
|
52
|
+
try:
|
|
53
|
+
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
|
54
|
+
except Exception:
|
|
55
|
+
pass
|
|
56
|
+
ap = build_parser()
|
|
57
|
+
args = ap.parse_args(argv)
|
|
58
|
+
|
|
59
|
+
if len(args.csvs) < 2:
|
|
60
|
+
print("[FAIL] at least two CSVs are required")
|
|
61
|
+
return 1
|
|
62
|
+
|
|
63
|
+
names = [os.path.splitext(os.path.basename(p))[0] for p in args.csvs]
|
|
64
|
+
data = [load(p) for p in args.csvs]
|
|
65
|
+
|
|
66
|
+
base_terms = set(data[0])
|
|
67
|
+
for i in range(1, len(data)):
|
|
68
|
+
other = set(data[i])
|
|
69
|
+
if base_terms != other:
|
|
70
|
+
print(f"[WARN] {names[0]} vs {names[i]} term sets differ: "
|
|
71
|
+
f"ref-only {sorted(base_terms - other)} | {names[i]}-only {sorted(other - base_terms)}")
|
|
72
|
+
common = sorted(base_terms & set.intersection(*[set(d) for d in data[1:]]))
|
|
73
|
+
if not common:
|
|
74
|
+
print("[FAIL] no common terms")
|
|
75
|
+
return 1
|
|
76
|
+
|
|
77
|
+
pairs = [(0, i) for i in range(1, len(data))]
|
|
78
|
+
print(f"reference: {names[0]} | checking: {[names[i] for _, i in pairs]}")
|
|
79
|
+
print(f"{'term':<24}" + "".join(f"{names[0]}vs{names[i]:<10}" for _, i in pairs))
|
|
80
|
+
print("-" * (24 + 20 * len(pairs)))
|
|
81
|
+
|
|
82
|
+
worst = 0
|
|
83
|
+
for t in common:
|
|
84
|
+
cells = []
|
|
85
|
+
for b, i in pairs:
|
|
86
|
+
g, dc, ds, dp = grade(*data[b][t], *data[i][t], args)
|
|
87
|
+
worst = max(worst, GRADES[g])
|
|
88
|
+
cells.append(g)
|
|
89
|
+
print(f"{t:<24}" + "".join(f"{c:<20}" for c in cells))
|
|
90
|
+
|
|
91
|
+
print("-" * (24 + 20 * len(pairs)))
|
|
92
|
+
for b, i in pairs:
|
|
93
|
+
stats = {"bit-exact": 0, "aligned": 0, "FAIL": 0}
|
|
94
|
+
for t in common:
|
|
95
|
+
g, *_ = grade(*data[b][t], *data[i][t], args)
|
|
96
|
+
stats[g] += 1
|
|
97
|
+
print(f"{names[0]} vs {names[i]}: bit-exact {stats['bit-exact']} | "
|
|
98
|
+
f"aligned {stats['aligned']} | FAIL {stats['FAIL']} (of {len(common)})")
|
|
99
|
+
|
|
100
|
+
if worst == GRADES["FAIL"]:
|
|
101
|
+
print("\nRESULT: FAIL — stop interpreting; debug sample filters / weights / cluster level / vcov first")
|
|
102
|
+
return 1
|
|
103
|
+
lvl = "bit-exact" if worst == 0 else "aligned"
|
|
104
|
+
print(f"\nRESULT: PASS ({lvl}) — all implementations agree "
|
|
105
|
+
f"(bit-exact = machine precision, aligned = within tolerance); "
|
|
106
|
+
f"identification strategy still needs human review")
|
|
107
|
+
return 0
|
|
108
|
+
|
|
109
|
+
|
|
110
|
+
if __name__ == "__main__":
|
|
111
|
+
sys.exit(main())
|
|
@@ -0,0 +1,198 @@
|
|
|
1
|
+
# -*- coding: utf-8 -*-
|
|
2
|
+
"""
|
|
3
|
+
report.py — EconWorkbench report module: results CSV -> journal-ready three-line tables
|
|
4
|
+
Input : one or more crosscheck-contract CSVs (columns: term,estimate,se,pvalue)
|
|
5
|
+
Output: LaTeX (booktabs three-line), Word (docx three-line), PNG preview
|
|
6
|
+
"""
|
|
7
|
+
import sys, os, csv, argparse
|
|
8
|
+
|
|
9
|
+
NOTES = "Standard errors in parentheses. *** p<0.01, ** p<0.05, * p<0.1"
|
|
10
|
+
|
|
11
|
+
|
|
12
|
+
def build_parser():
|
|
13
|
+
ap = argparse.ArgumentParser(
|
|
14
|
+
prog="econ-report",
|
|
15
|
+
description="Turn result CSVs (term,estimate,se,pvalue) into journal-ready three-line tables.")
|
|
16
|
+
ap.add_argument("csvs", nargs="+")
|
|
17
|
+
ap.add_argument("--title", default="")
|
|
18
|
+
ap.add_argument("--out", default="table")
|
|
19
|
+
ap.add_argument("--format", default="tex,docx,png")
|
|
20
|
+
ap.add_argument("--decimals", type=int, default=3)
|
|
21
|
+
return ap
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
def load(path):
|
|
25
|
+
rows = []
|
|
26
|
+
with open(path, newline="", encoding="utf-8-sig") as f:
|
|
27
|
+
for r in csv.DictReader(f):
|
|
28
|
+
rows.append((r["term"].strip(), float(r["estimate"]), float(r["se"]), float(r["pvalue"])))
|
|
29
|
+
return rows
|
|
30
|
+
|
|
31
|
+
|
|
32
|
+
def stars(p):
|
|
33
|
+
return "***" if p < 0.01 else ("**" if p < 0.05 else ("*" if p < 0.1 else ""))
|
|
34
|
+
|
|
35
|
+
|
|
36
|
+
def main(argv=None):
|
|
37
|
+
try:
|
|
38
|
+
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
|
39
|
+
except Exception:
|
|
40
|
+
pass
|
|
41
|
+
args = build_parser().parse_args(argv)
|
|
42
|
+
|
|
43
|
+
models = []
|
|
44
|
+
for p in args.csvs:
|
|
45
|
+
models.append((os.path.splitext(os.path.basename(p))[0], load(p)))
|
|
46
|
+
terms = [t for t, *_ in models[0][1]]
|
|
47
|
+
d = args.decimals
|
|
48
|
+
fmts = args.format.split(",")
|
|
49
|
+
|
|
50
|
+
# ---------- LaTeX ----------
|
|
51
|
+
if "tex" in fmts:
|
|
52
|
+
tex = ["\\begin{table}[htbp]", "\\centering"]
|
|
53
|
+
if args.title:
|
|
54
|
+
tex.append("\\caption{" + args.title + "}")
|
|
55
|
+
tex.append("\\begin{tabular}{l" + "c" * len(models) + "}")
|
|
56
|
+
tex.append("\\toprule")
|
|
57
|
+
tex.append(" & " + " & ".join(f"({i+1})" for i in range(len(models))) + " \\\\")
|
|
58
|
+
tex.append("\\midrule")
|
|
59
|
+
for t in terms:
|
|
60
|
+
cells = [f"{row[1]:.{d}f}{stars(row[3])}"
|
|
61
|
+
if (row := next((r for r in rows if r[0] == t), None)) else ""
|
|
62
|
+
for _, rows in models]
|
|
63
|
+
tex.append(f"{t} & " + " & ".join(cells) + " \\\\")
|
|
64
|
+
ses = [f"({row[2]:.{d}f})" if (row := next((r for r in rows if r[0] == t), None)) else ""
|
|
65
|
+
for _, rows in models]
|
|
66
|
+
if any(ses):
|
|
67
|
+
tex.append(" & " + " & ".join(ses) + " \\\\")
|
|
68
|
+
tex.append("\\midrule")
|
|
69
|
+
tex.append(" & " + " & ".join(m[0] for m in models) + " \\\\")
|
|
70
|
+
tex.append("\\bottomrule")
|
|
71
|
+
tex.append("\\end{tabular}")
|
|
72
|
+
tex.append("\\begin{flushleft}\\footnotesize " + NOTES.replace("<", "$<$") + "\\end{flushleft}")
|
|
73
|
+
tex.append("\\end{table}")
|
|
74
|
+
with open(args.out + ".tex", "w", encoding="utf-8") as f:
|
|
75
|
+
f.write("\n".join(tex) + "\n")
|
|
76
|
+
print("wrote", args.out + ".tex")
|
|
77
|
+
|
|
78
|
+
# ---------- Word (three-line style) ----------
|
|
79
|
+
if "docx" in fmts:
|
|
80
|
+
from docx import Document
|
|
81
|
+
from docx.shared import Pt
|
|
82
|
+
from docx.enum.text import WD_ALIGN_PARAGRAPH
|
|
83
|
+
from docx.enum.table import WD_TABLE_ALIGNMENT
|
|
84
|
+
from docx.oxml.ns import qn
|
|
85
|
+
from docx.oxml import OxmlElement
|
|
86
|
+
|
|
87
|
+
doc = Document()
|
|
88
|
+
style = doc.styles["Normal"]
|
|
89
|
+
style.font.name = "Times New Roman"
|
|
90
|
+
style._element.rPr.rFonts.set(qn("w:eastAsia"), "宋体")
|
|
91
|
+
style.font.size = Pt(10.5)
|
|
92
|
+
|
|
93
|
+
if args.title:
|
|
94
|
+
cap = doc.add_paragraph()
|
|
95
|
+
cap.alignment = WD_ALIGN_PARAGRAPH.CENTER
|
|
96
|
+
r = cap.add_run(args.title)
|
|
97
|
+
r.font.bold = True
|
|
98
|
+
|
|
99
|
+
tbl = doc.add_table(rows=1 + 2 * len(terms) + 1, cols=1 + len(models))
|
|
100
|
+
tbl.alignment = WD_TABLE_ALIGNMENT.CENTER
|
|
101
|
+
|
|
102
|
+
def set_borders(cell, top=None, bottom=None):
|
|
103
|
+
tcPr = cell._tc.get_or_add_tcPr()
|
|
104
|
+
borders = OxmlElement("w:tcBorders")
|
|
105
|
+
for edge, sz in (("top", top), ("bottom", bottom)):
|
|
106
|
+
if sz:
|
|
107
|
+
el = OxmlElement(f"w:{edge}")
|
|
108
|
+
el.set(qn("w:val"), "single")
|
|
109
|
+
el.set(qn("w:sz"), str(sz))
|
|
110
|
+
el.set(qn("w:color"), "000000")
|
|
111
|
+
borders.append(el)
|
|
112
|
+
tcPr.append(borders)
|
|
113
|
+
|
|
114
|
+
def put(cell, text, bold=False, center=True):
|
|
115
|
+
p = cell.paragraphs[0]
|
|
116
|
+
p.alignment = WD_ALIGN_PARAGRAPH.CENTER if center else WD_ALIGN_PARAGRAPH.LEFT
|
|
117
|
+
r = p.add_run(text)
|
|
118
|
+
r.font.bold = bold
|
|
119
|
+
r.font.size = Pt(10.5)
|
|
120
|
+
|
|
121
|
+
hdr = tbl.rows[0].cells
|
|
122
|
+
for j in range(1 + len(models)):
|
|
123
|
+
put(hdr[j], f"({j})" if j else "", bold=True)
|
|
124
|
+
set_borders(hdr[j], top=12, bottom=6)
|
|
125
|
+
|
|
126
|
+
for i, t in enumerate(terms):
|
|
127
|
+
rc = tbl.rows[1 + 2 * i].cells
|
|
128
|
+
put(rc[0], t, center=False)
|
|
129
|
+
for j, (name, rows) in enumerate(models):
|
|
130
|
+
row = next((r for r in rows if r[0] == t), None)
|
|
131
|
+
put(rc[j + 1], (f"{row[1]:.{d}f}{stars(row[3])}") if row else "")
|
|
132
|
+
sc = tbl.rows[2 + 2 * i].cells
|
|
133
|
+
put(sc[0], "")
|
|
134
|
+
for j, (name, rows) in enumerate(models):
|
|
135
|
+
row = next((r for r in rows if r[0] == t), None)
|
|
136
|
+
put(sc[j + 1], f"({row[2]:.{d}f})" if row else "")
|
|
137
|
+
|
|
138
|
+
last = tbl.rows[-1].cells
|
|
139
|
+
put(last[0], "", center=False)
|
|
140
|
+
for j, (name, _) in enumerate(models):
|
|
141
|
+
put(last[j + 1], name)
|
|
142
|
+
set_borders(last[j + 1], bottom=12)
|
|
143
|
+
set_borders(last[0], bottom=12)
|
|
144
|
+
|
|
145
|
+
note_p = doc.add_paragraph()
|
|
146
|
+
nr = note_p.add_run(NOTES)
|
|
147
|
+
nr.font.size = Pt(8.5)
|
|
148
|
+
|
|
149
|
+
doc.save(args.out + ".docx")
|
|
150
|
+
print("wrote", args.out + ".docx")
|
|
151
|
+
|
|
152
|
+
# ---------- PNG preview ----------
|
|
153
|
+
if "png" in fmts:
|
|
154
|
+
import matplotlib
|
|
155
|
+
matplotlib.use("Agg")
|
|
156
|
+
import matplotlib.pyplot as plt
|
|
157
|
+
|
|
158
|
+
n_rows = 1 + 2 * len(terms) + 1
|
|
159
|
+
cell_text = []
|
|
160
|
+
for t in terms:
|
|
161
|
+
coef = [f"{row[1]:.{d}f}{stars(row[3])}" if (row := next((r for r in rows if r[0] == t), None)) else ""
|
|
162
|
+
for _, rows in models]
|
|
163
|
+
ses = [f"({row[2]:.{d}f})" if (row := next((r for r in rows if r[0] == t), None)) else ""
|
|
164
|
+
for _, rows in models]
|
|
165
|
+
cell_text.append([t] + coef)
|
|
166
|
+
cell_text.append([""] + ses)
|
|
167
|
+
header = [""] + [f"({j+1})" for j in range(len(models))]
|
|
168
|
+
footer = [""] + [name for name, _ in models]
|
|
169
|
+
|
|
170
|
+
fig, ax = plt.subplots(figsize=(2.2 + 1.9 * len(models), 0.34 * n_rows + 1.1))
|
|
171
|
+
ax.set_xlim(0, 1 + len(models))
|
|
172
|
+
ax.set_ylim(-1.3, n_rows + 1)
|
|
173
|
+
ax.axis("off")
|
|
174
|
+
if args.title:
|
|
175
|
+
ax.set_title(args.title, fontsize=12, pad=14)
|
|
176
|
+
|
|
177
|
+
def draw_row(y, vals, bold=False, size=10):
|
|
178
|
+
for j, v in enumerate(vals):
|
|
179
|
+
ax.text(j + 0.02, y, v, fontsize=size,
|
|
180
|
+
fontweight="bold" if bold else "normal",
|
|
181
|
+
ha="left", va="center", family="serif")
|
|
182
|
+
|
|
183
|
+
draw_row(n_rows + 0.35, header, bold=True)
|
|
184
|
+
for i, row_vals in enumerate(cell_text):
|
|
185
|
+
draw_row(n_rows - 0.55 - i, row_vals)
|
|
186
|
+
draw_row(0.15, footer)
|
|
187
|
+
ax.plot([0, 1 + len(models)], [n_rows + 0.75, n_rows + 0.75], color="k", lw=1.4)
|
|
188
|
+
ax.plot([0, 1 + len(models)], [n_rows + 0.05, n_rows + 0.05], color="k", lw=0.7)
|
|
189
|
+
ax.plot([0, 1 + len(models)], [-0.2, -0.2], color="k", lw=1.4)
|
|
190
|
+
ax.text(0, -0.75, NOTES, fontsize=8, ha="left", va="center", family="serif")
|
|
191
|
+
fig.savefig(args.out + ".png", dpi=200, bbox_inches="tight")
|
|
192
|
+
print("wrote", args.out + ".png")
|
|
193
|
+
|
|
194
|
+
return 0
|
|
195
|
+
|
|
196
|
+
|
|
197
|
+
if __name__ == "__main__":
|
|
198
|
+
sys.exit(main())
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: econworkbench
|
|
3
|
+
Version: 1.2.0
|
|
4
|
+
Summary: From idea to referee-ready: an open workbench for empirical research. Core: crosscheck (R/Python/Stata triple validation) + report (journal-ready three-line tables).
|
|
5
|
+
Author: ljftwq-dev
|
|
6
|
+
License: MIT License
|
|
7
|
+
|
|
8
|
+
Copyright (c) 2026 ljftwq-dev
|
|
9
|
+
|
|
10
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
11
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
12
|
+
in the Software without restriction, including without limitation the rights
|
|
13
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
14
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
15
|
+
furnished to do so, subject to the following conditions:
|
|
16
|
+
|
|
17
|
+
The above copyright notice and this permission notice shall be included in all
|
|
18
|
+
copies or substantial portions of the Software.
|
|
19
|
+
|
|
20
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
21
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
22
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
23
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
24
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
25
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
26
|
+
SOFTWARE.
|
|
27
|
+
|
|
28
|
+
Keywords: econometrics,causal-inference,difference-in-differences,stata,r,reproducibility
|
|
29
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
30
|
+
Classifier: Programming Language :: Python :: 3
|
|
31
|
+
Classifier: Topic :: Scientific/Engineering
|
|
32
|
+
Requires-Python: >=3.9
|
|
33
|
+
Description-Content-Type: text/markdown
|
|
34
|
+
License-File: LICENSE
|
|
35
|
+
Provides-Extra: report
|
|
36
|
+
Requires-Dist: python-docx; extra == "report"
|
|
37
|
+
Requires-Dist: matplotlib; extra == "report"
|
|
38
|
+
Dynamic: license-file
|
|
39
|
+
|
|
40
|
+
# EconWorkbench
|
|
41
|
+
|
|
42
|
+
[](https://github.com/ljftwq-dev/EconWorkbench/actions/workflows/ci.yml)
|
|
43
|
+
[](LICENSE)
|
|
44
|
+
|
|
45
|
+
[English](README.md) | [中文](README_CN.md)
|
|
46
|
+
|
|
47
|
+
**From idea to referee-ready — an open workbench for empirical research.**
|
|
48
|
+
|
|
49
|
+
EconWorkbench is a toolkit for the empirical-research workflow: literature, data, design, estimation, **verification**, and reporting. Research judgment stays with you — the workbench makes each step faster and the numbers trustworthy.
|
|
50
|
+
|
|
51
|
+
**v1.0 ships the core: `crosscheck/`** — run the *same* model in **R, Python, and Stata**, then diff the results automatically. If the numbers agree, your code is (almost certainly) right. If they don't, you stop interpreting — before a reviewer (or a referee report) stops you.
|
|
52
|
+
|
|
53
|
+
<p align="center">
|
|
54
|
+
<img src="assets/triple_crosscheck_cs2021.png" alt="R, Python and Stata terminals side by side reproducing Callaway-Sant'Anna (2021), all showing ATT = -0.0399513" width="900">
|
|
55
|
+
</p>
|
|
56
|
+
|
|
57
|
+
*Live screenshot: Callaway & Sant'Anna (2021) staggered DiD on `mpdta`, run independently in R (`did`), Python (`StatsPAI`), and Stata (`csdid`). All three report Overall ATT = **-0.0399513**.*
|
|
58
|
+
|
|
59
|
+
## Why
|
|
60
|
+
|
|
61
|
+
AI agents write most of our econometrics code now. But an LLM that writes the wrong `vcov` in two languages — consistently — will hand you confident nonsense. The fix is older than LLMs: **independent implementations of the same estimator should agree**.
|
|
62
|
+
|
|
63
|
+
- R's `did` (Callaway et al.), Stata's `csdid` (Sant'Anna et al.), and Python ports are written by different teams. Same data + same model + same numbers = implementation is clean.
|
|
64
|
+
- One run, one number, no cross-check = you cannot tell a result from a bug.
|
|
65
|
+
|
|
66
|
+
This is the "double-entry bookkeeping" for regressions — inspired by a 2026 lecture series on AI agents for social-science research (where the speaker, priced out of Stata licenses, could only cross-check R vs Python), and by the [StatsPAI](https://github.com/brycewang-stanford/StatsPAI) parity index, which grades its own estimators against R/Stata references.
|
|
67
|
+
|
|
68
|
+
## Quick start
|
|
69
|
+
|
|
70
|
+
Core is zero-dependency (pure stdlib):
|
|
71
|
+
|
|
72
|
+
```
|
|
73
|
+
pip install econworkbench # or: pip install .[report] for tables
|
|
74
|
+
|
|
75
|
+
# 1. Run the same model in R / Python / Stata; each script writes a 4-column CSV:
|
|
76
|
+
# term, estimate, se, pvalue
|
|
77
|
+
Rscript my_model.R # -> r_results.csv
|
|
78
|
+
python my_model.py # -> py_results.csv
|
|
79
|
+
"C:\Program Files\Stata19\StataSE-64.exe" /e do my_model.do # -> stata_results.csv
|
|
80
|
+
|
|
81
|
+
# 2. Diff them (first CSV is the reference):
|
|
82
|
+
econ-crosscheck r_results.csv py_results.csv stata_results.csv
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
Output:
|
|
86
|
+
|
|
87
|
+
```
|
|
88
|
+
term r_resultsvspy_resultsr_resultsvsstata_results
|
|
89
|
+
--------------------------------------------------------------------
|
|
90
|
+
g2004_t2004 bit-exact aligned
|
|
91
|
+
...
|
|
92
|
+
overall_simple bit-exact aligned
|
|
93
|
+
--------------------------------------------------------------------
|
|
94
|
+
r_results vs py_results: bit-exact 8 | aligned 0 | FAIL 0
|
|
95
|
+
r_results vs stata_results: bit-exact 0 | aligned 8 | FAIL 0
|
|
96
|
+
RESULT: PASS (aligned) — exit code 0
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
## Verdicts
|
|
100
|
+
|
|
101
|
+
| Verdict | Meaning | Default tolerance |
|
|
102
|
+
|---|---|---|
|
|
103
|
+
| `bit-exact` | Machine-precision agreement | Δ ≤ 1e-10 |
|
|
104
|
+
| `aligned` | Agreement within tolerance | Δcoef ≤ 1e-6, ΔSE/SE ≤ 1e-4, Δp ≤ 1e-4 |
|
|
105
|
+
| `FAIL` | Significance flips or tolerance breached | — exit code 1 |
|
|
106
|
+
|
|
107
|
+
Exit codes make it gate-able: wire it into CI, a Makefile, or your SDD pipeline as a release gate.
|
|
108
|
+
|
|
109
|
+
## Example: Callaway & Sant'Anna (2021) replication
|
|
110
|
+
|
|
111
|
+
[`examples/cs2021_mpdta/`](examples/cs2021_mpdta/) replicates the canonical staggered-DiD application (minimum-wage effects on teen employment, 2,500 county-year obs) three ways:
|
|
112
|
+
|
|
113
|
+
| | Implementation | Authors' own package |
|
|
114
|
+
|---|---|---|
|
|
115
|
+
| R 4.6.1 | `did::att_gt()` + `aggte()` | Callaway & Sant'Anna |
|
|
116
|
+
| Python 3.14 | `StatsPAI.callaway_santanna()` | port with its own parity evidence |
|
|
117
|
+
| StataNow 19.5 | `csdid` + `csdid_estat simple` | Sant'Anna, Goodman-Bacon & Pedro |
|
|
118
|
+
|
|
119
|
+
Result: **R ↔ Python bit-exact on all 8 quantities** (7 post-period ATT(g,t) + overall); **R ↔ Stata aligned** (Δ ≈ 1e-7, different DR-IPW optimizers). Overall ATT = -0.0399 matches the published paper.
|
|
120
|
+
|
|
121
|
+
Run it yourself: each folder contains the three scripts, the shared dataset (`mpdta_data.csv` — one source of truth, all three languages read the same file), and the three result CSVs.
|
|
122
|
+
|
|
123
|
+
## From estimates to tables (`report/`, v1.2)
|
|
124
|
+
|
|
125
|
+
The same CSVs that feed `crosscheck` also feed `report` — one command from estimates to a journal-ready three-line table:
|
|
126
|
+
|
|
127
|
+
```
|
|
128
|
+
econ-report r_results.csv py_results.csv stata_results.csv --title "Table 1" --out table1
|
|
129
|
+
# -> table1.tex (booktabs) + table1.docx (Word three-line) + table1.png (preview)
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
<p align="center">
|
|
133
|
+
<img src="examples/cs2021_mpdta/table1.png" alt="Three-line regression table comparing R/Python/Stata estimates" width="640">
|
|
134
|
+
</p>
|
|
135
|
+
|
|
136
|
+
Significance stars, SEs in parentheses, top/mid/bottom rules — in all three formats. Column labels sit at the bottom, `esttab`-style.
|
|
137
|
+
|
|
138
|
+
## What it does *not* do
|
|
139
|
+
|
|
140
|
+
- It cannot validate your **identification strategy** — two implementations of a wrong model agree perfectly. (The companion `REVIEWER_CHECKLIST.md` in the parent project covers clustering levels, staggered-DiD pitfalls, pre-trends, etc.)
|
|
141
|
+
- It is not a paper factory. Judgment stays with the researcher.
|
|
142
|
+
|
|
143
|
+
## Related work
|
|
144
|
+
|
|
145
|
+
- [StatsPAI](https://github.com/brycewang-stanford/StatsPAI) — agent-native Stata/R replacement with a queryable parity index (a superset of this idea, inside one library).
|
|
146
|
+
- Sakana AI-Scientist, Agent Laboratory — end-to-end "AI scientist" pipelines (the opposite bet: automation over verification).
|
|
147
|
+
|
|
148
|
+
## License
|
|
149
|
+
|
|
150
|
+
MIT
|
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
README.md
|
|
3
|
+
pyproject.toml
|
|
4
|
+
src/econworkbench/__init__.py
|
|
5
|
+
src/econworkbench/crosscheck.py
|
|
6
|
+
src/econworkbench/report.py
|
|
7
|
+
src/econworkbench.egg-info/PKG-INFO
|
|
8
|
+
src/econworkbench.egg-info/SOURCES.txt
|
|
9
|
+
src/econworkbench.egg-info/dependency_links.txt
|
|
10
|
+
src/econworkbench.egg-info/entry_points.txt
|
|
11
|
+
src/econworkbench.egg-info/requires.txt
|
|
12
|
+
src/econworkbench.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
econworkbench
|