ltc-paper-scripts 0.1.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (35) hide show
  1. ltc_paper_scripts/README.md +87 -0
  2. ltc_paper_scripts/TICK_ALGORITHM_LICENSE.txt +26 -0
  3. ltc_paper_scripts/build_latex.py +293 -0
  4. ltc_paper_scripts/data/codex_synthetic/new_release_preview.xlsx +0 -0
  5. ltc_paper_scripts/exhibit_selection.py +52 -0
  6. ltc_paper_scripts/figure_ticks.py +51 -0
  7. ltc_paper_scripts/fonts/DejaVuSans-Bold.ttf +0 -0
  8. ltc_paper_scripts/fonts/DejaVuSans.ttf +0 -0
  9. ltc_paper_scripts/fonts/LICENSE.txt +96 -0
  10. ltc_paper_scripts/latex_figures.py +250 -0
  11. ltc_paper_scripts/latex_requirements.txt +4 -0
  12. ltc_paper_scripts/latex_source/outputs/tables/output/age25_earnings_table.tex +38 -0
  13. ltc_paper_scripts/latex_source/outputs/tables/output/balance_table.tex +54 -0
  14. ltc_paper_scripts/latex_source/outputs/tables/output/college_data_comparison_table.tex +20 -0
  15. ltc_paper_scripts/latex_source/outputs/tables/output/college_enrollment_completion_table.tex +31 -0
  16. ltc_paper_scripts/latex_source/outputs/tables/output/college_heterogeneity_table.tex +36 -0
  17. ltc_paper_scripts/latex_source/outputs/tables/output/college_mediation_additional_specs_table.tex +30 -0
  18. ltc_paper_scripts/latex_source/outputs/tables/output/college_test_score_mediation_table.tex +18 -0
  19. ltc_paper_scripts/latex_source/outputs/tables/output/earnings_heterogeneity_table.tex +54 -0
  20. ltc_paper_scripts/latex_source/outputs/tables/output/earnings_robustness_table.tex +29 -0
  21. ltc_paper_scripts/latex_source/outputs/tables/output/earnings_threshold_table.tex +27 -0
  22. ltc_paper_scripts/latex_source/outputs/tables/output/other_mediators_table.tex +31 -0
  23. ltc_paper_scripts/latex_source/outputs/tables/output/peer_family_comparisons_table.tex +20 -0
  24. ltc_paper_scripts/latex_source/outputs/tables/output/summary_statistics_table.tex +60 -0
  25. ltc_paper_scripts/latex_source/paper/appendix.tex +138 -0
  26. ltc_paper_scripts/latex_source/paper/preamble.tex +502 -0
  27. ltc_paper_scripts/latex_source/paper/tables-and-figures.tex +147 -0
  28. ltc_paper_scripts/mappings.py +328 -0
  29. ltc_paper_scripts/paths.py +23 -0
  30. ltc_paper_scripts/run_latex.sh +4 -0
  31. ltc_paper_scripts/split_inputs.py +163 -0
  32. ltc_paper_scripts/table_bindings.json +4084 -0
  33. ltc_paper_scripts-0.1.0.dist-info/METADATA +94 -0
  34. ltc_paper_scripts-0.1.0.dist-info/RECORD +35 -0
  35. ltc_paper_scripts-0.1.0.dist-info/WHEEL +4 -0
@@ -0,0 +1,87 @@
1
+ # Paper PDF build scripts - November 4, 2026
2
+
3
+ Standalone snapshot of the approved 17-page paper exhibit builder. Copy this
4
+ entire folder into Census. It does not import anything from the parent
5
+ `ltc_code` package or the separate `paper/` folder.
6
+
7
+ This folder is distributed as `ltc-paper-scripts` 0.1.0. It contains the paper
8
+ scripts and assets only. Installing it has no declared dependencies and does
9
+ not install rendering libraries. There is no `pyproject.toml`, `__init__.py`,
10
+ `uv.lock`, or `.gitignore` in the download. The wheel includes only the required
11
+ PyPI distribution metadata in addition to this folder.
12
+
13
+ To locate the installed folder without an initialization file:
14
+
15
+ ```python
16
+ import ltc_paper_scripts
17
+ print(next(iter(ltc_paper_scripts.__path__)))
18
+ ```
19
+
20
+ Copy the whole folder into your editable Census code directory. The normal
21
+ command `python build_latex.py` uses the libraries already in that environment.
22
+ The requirements file and Bash runner are optional conveniences for setting up
23
+ an environment where dependency installation is permitted.
24
+
25
+ ## Run in Census
26
+
27
+ Use Python 3.9 or newer, the Python libraries in `latex_requirements.txt`, and
28
+ **pdfLaTeX** with the TeX packages listed in `latex_source/paper/preamble.tex`.
29
+ The build requires no R, XeLaTeX, LuaLaTeX, shell escape, or internet access
30
+ once those dependencies are installed. It compiles three pdfLaTeX passes.
31
+
32
+ 1. Supply your real results workbook. The included workbook combines existing
33
+ disclosed/example values and synthetic examples; it is not a new release.
34
+ 2. Edit `paths.py`: set `PENDING_INPUT`, the output location, and `PDFLATEX` if
35
+ the executable is not on PATH. Keep `PAPER_ROOT` at this folder, or adjust
36
+ it when using a shared `paths.py` in a separate code directory.
37
+ 3. Edit `input_mapping()` in `build_latex.py`: sheet/outcome/header names,
38
+ sample/specification names, the network roster, units, and benchmarks as
39
+ needed. Preserve the descriptive mapping keys on the left.
40
+ 4. From this folder, run:
41
+
42
+ ```sh
43
+ python build_latex.py
44
+ ```
45
+
46
+ On a machine with `uv`, the equivalent command is:
47
+
48
+ ```sh
49
+ uv run --no-project --with-requirements latex_requirements.txt python build_latex.py
50
+ ```
51
+
52
+ `run_latex.sh` is an optional convenience for systems with Bash. Census does
53
+ not need Bash or `uv` if Python and the required libraries are already available.
54
+
55
+ The output is `output/charter_exhibits_latex.pdf`, accompanied by an audit JSON.
56
+ The `output/latex_build/` directory contains the generated TeX and compile logs.
57
+ There is no preview banner on the PDF pages.
58
+
59
+ ## Included exhibits and calculations
60
+
61
+ - Figures 1-3, Tables 1-7, Figures A2-A4, and Tables A1-A4.
62
+ - Excludes Figure 4, Figure A1, Tables A5-A7, and Table 1 Panel E.
63
+ - All results default to the new workbook. Table results are red; figures
64
+ preserve their orange/teal colors.
65
+ - Wage percent = `100 * wage coef / wage CCM`. Table 6 uses this same unrounded
66
+ percentage for every mediation denominator; there is no fixed 13.6 value.
67
+ - Real inputs must contain the required results in the mapped units. This
68
+ builder consumes estimates, standard errors, control means, counts, and
69
+ summary statistics; it does not estimate regressions from microdata.
70
+
71
+ ## Files to keep together
72
+
73
+ Keep every included Python file, `table_bindings.json`, the entire
74
+ `latex_source/` tree, and `fonts/`. These are the core renderer and its assets.
75
+ Only `paths.py` and `build_latex.py` need routine configuration. The example
76
+ workbook can be replaced with real results. The old disclosed workbook is not
77
+ required unless you explicitly opt into `reuse_disclosed`.
78
+
79
+ The open DejaVu fonts and their license are bundled for portability. Installed
80
+ Lucida Sans is preferred where available; otherwise typography uses DejaVu.
81
+ Proprietary Office fonts are not included. The tick algorithm's license is in
82
+ `TICK_ALGORITHM_LICENSE.txt`.
83
+
84
+ Copied from `src/ltc_code/paper/` on November 4, 2026. Runtime files and templates
85
+ match that approved builder; the README is specific to this standalone copy.
86
+ The requirements file permits ReportLab 4.5 and newer. On Python 3.9 it selects
87
+ ReportLab 4.x because ReportLab 5.1 requires Python 3.10 at runtime.
@@ -0,0 +1,26 @@
1
+ Copyright (c) 2016, Hassan Kibirige
2
+ All rights reserved.
3
+
4
+ Redistribution and use in source and binary forms, with or without
5
+ modification, are permitted provided that the following conditions are met:
6
+
7
+ * Redistributions of source code must retain the above copyright notice,
8
+ this list of conditions and the following disclaimer.
9
+ * Redistributions in binary form must reproduce the above copyright
10
+ notice, this list of conditions and the following disclaimer in the
11
+ documentation and/or other materials provided with the distribution.
12
+ * Neither the name of nor the names of its contributors may be used to
13
+ endorse or promote products derived from this software without specific
14
+ prior written permission.
15
+
16
+ THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
17
+ AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
18
+ IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
19
+ ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE
20
+ LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
21
+ CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
22
+ SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
23
+ INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
24
+ CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
25
+ ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
26
+ POSSIBILITY OF SUCH DAMAGE.
@@ -0,0 +1,293 @@
1
+ # --- Import necessary packages ---
2
+ import hashlib
3
+ import json
4
+ import re
5
+ import shutil
6
+ import subprocess
7
+ import sys
8
+ from pathlib import Path
9
+
10
+ # Run from your code folder, where the shared paths.py lives.
11
+ # Replace "." with that folder's absolute path if you run from elsewhere.
12
+ sys.path.append(".")
13
+ from paths import *
14
+ from mappings import resolve_inputs
15
+ from split_inputs import SplitInputs, build_tables
16
+ from latex_figures import make_figures, FIGURE_EXHIBITS
17
+ from exhibit_selection import select_exhibits
18
+ from pypdf import PdfReader
19
+
20
+ ###########################################################
21
+ # Edit workbook mappings here. Do not edit renderer or TeX files.
22
+ ###########################################################
23
+
24
+ COLOR_PENDING = True # Red pending table values; figures retain blue/orange.
25
+ # Keep manuscript numbering even when exhibits are omitted.
26
+ EXCLUDE_TABLES = ['lottery_sample_selection_table', 'cmo_details_table', 'intervention_calculation_table']
27
+ EXCLUDE_FIGURES = ['Figure 4', 'Figure A.1']
28
+
29
+
30
+ def input_mapping():
31
+ """Edit only the sheet names and variable names below for your Census file.
32
+
33
+ Paths are in paths.py. All results default to the pending workbook; reuse_disclosed opts into old results.
34
+ Keep the descriptive labels on the LEFT; edit the pair on the RIGHT.
35
+ Each pair is (sheet, variable). Use a quoted sheet name for any exception.
36
+ """
37
+ SUMMARY = 'summary_stats'
38
+ BALANCE = 'balance'
39
+ EFFECTS = 'TOT'
40
+ OLS = 'OLS'
41
+ FIRST_STAGE = 'first_stage'
42
+ ITT = 'ITT'
43
+ CONVERSIONS = 'mediation_conversions'
44
+ NETWORKS = 'TOT_by_cmo'
45
+ PREDICTIONS = 'predictions' # Change to 'TOTE' if predictions live there.
46
+
47
+ inputs = {
48
+ # Table 1: summary statistics
49
+ 'Summary: parent income': (SUMMARY, 'parent_income'),
50
+ 'Summary: parent inc rank': (SUMMARY, 'parent_inc_rank'),
51
+ 'Summary: female': (SUMMARY, 'female'),
52
+ 'Summary: hispanic': (SUMMARY, 'hispanic'),
53
+ 'Summary: black': (SUMMARY, 'black'),
54
+ 'Summary: white': (SUMMARY, 'white'),
55
+ 'Summary: entry age': (SUMMARY, 'entry_age'),
56
+ 'Summary: enrollment year': (SUMMARY, 'enrollment_year'),
57
+ 'Summary: enrollment years': (SUMMARY, 'enrollment_years'),
58
+ 'Summary: Age 25 earnings': (SUMMARY, 'wages25'),
59
+ 'Summary: 1098T college by 20': (SUMMARY, 'any1098T_by20'),
60
+ 'Summary: College attendance by 20': (SUMMARY, 'att_any_by_age20'),
61
+ 'Summary: employed25': (SUMMARY, 'employed25'),
62
+ 'Summary: n ppl': (SUMMARY, 'n_ppl'),
63
+ # Table 2: balance
64
+ 'Balance: black': (BALANCE, 'black'),
65
+ 'Balance: hispanic': (BALANCE, 'hispanic'),
66
+ 'Balance: white': (BALANCE, 'white'),
67
+ 'Balance: female': (BALANCE, 'female'),
68
+ 'Balance: parent inc rank': (BALANCE, 'parent_inc_rank'),
69
+ 'Balance: med hhinc2000': (BALANCE, 'med_hhinc2000'),
70
+ 'Balance: wages25 pre treatment': (BALANCE, 'wages25_pre_treatment'),
71
+ 'Balance: linked tax': (BALANCE, 'linked_tax'),
72
+ 'Balance: joint covariate test': (BALANCE, 'joint_covariates'),
73
+ # Effects shared across tables
74
+ 'Age 25 household income': (EFFECTS, 'agi25'),
75
+ 'Age 25 earnings': (EFFECTS, 'wages25'),
76
+ 'Surrogate earnings at 31 by subgroup': (EFFECTS, 'wages31_firm_chars'),
77
+ '1098T college by 20': (EFFECTS, 'any1098T_by20'),
78
+ 'BA completion by year 6': (EFFECTS, 'cmp_BA_byY6'),
79
+ 'College attendance by 20': (EFFECTS, 'att_any_by_age20'),
80
+ 'Two-year attendance by 20': (EFFECTS, 'att_2yr_by_age20'),
81
+ 'Four-year attendance by 20': (EFFECTS, 'att_4yr_by_age20'),
82
+ 'Elite college attendance by 20': (EFFECTS, 'att_elite_by_age20'),
83
+ 'Elite BA completion by year 6': (EFFECTS, 'cmp_elite_byY6'),
84
+ 'Firm selective BA rate': (EFFECTS, 'firm19_selective_ba'),
85
+ 'Firm worker earnings': (EFFECTS, 'firm19_wages30_40'),
86
+ 'Firm retention': (EFFECTS, 'firm19_retention'),
87
+ 'Age 25 spouse earnings': (EFFECTS, 'spouse_wages25'),
88
+ 'Married at 25': (EFFECTS, 'married25'),
89
+ 'Any completion by year 6': (EFFECTS, 'cmp_any_byY6'),
90
+ 'AA completion by year 6': (EFFECTS, 'cmp_AA_byY6'),
91
+ 'Predicted any completion': (EFFECTS, 'pred_cmp_any'),
92
+ 'Predicted four-year completion': (EFFECTS, 'pred_cmp_4yr'),
93
+ 'Predicted two-year completion': (EFFECTS, 'pred_cmp_2yr'),
94
+ 'Selective BA completion by year 6': (EFFECTS, 'cmp_selective_byY6'),
95
+ 'Figure A2: college at each age': (EFFECTS, 'att_any_age{age}'),
96
+ 'Figure A4: firm surrogate earnings': (EFFECTS, 'wages{age}_firm_chars'),
97
+ # Table A1: observational OLS comparison mean
98
+ 'OLS: Age 25 earnings': (OLS, 'wages25'),
99
+ # Table A2: predicted earnings
100
+ 'Predicted earnings above $20,000': (PREDICTIONS, 'wages25_over_20000'),
101
+ 'Predicted earnings above $50,000': (PREDICTIONS, 'wages25_over_50000'),
102
+ 'Predicted earnings above $100,000': (PREDICTIONS, 'wages25_over_100000'),
103
+ # Previously disclosed results and newly included figures
104
+ 'First stage: charter enrollment': (FIRST_STAGE, 'enrolled'),
105
+ 'ITT: Age 25 earnings': (ITT, 'wages25'),
106
+ 'Peers: parent income': (OLS, 'peer_parent_income'),
107
+ 'Teen birth by age 20': (EFFECTS, 'teenbirth_claim20'),
108
+ 'Actual earnings above $0': (EFFECTS, 'wages25_over_0'),
109
+ 'Actual earnings above $20,000': (EFFECTS, 'wages25_over_20000'),
110
+ 'Actual earnings above $50,000': (EFFECTS, 'wages25_over_50000'),
111
+ 'Actual earnings above $100,000': (EFFECTS, 'wages25_over_100000'),
112
+ 'Mediation: BA earnings conversion': (CONVERSIONS, 'cmp_BA_byY6'),
113
+ 'Mediation: selective BA earnings conversion': (CONVERSIONS, 'cmp_selective_byY6'),
114
+ 'Mediation: elite BA earnings conversion': (CONVERSIONS, 'cmp_elite_byY6'),
115
+ 'Figures 1/A4: observed earnings at each age': (EFFECTS, 'wages{age}'),
116
+ 'Figure 1: combined surrogate earnings': (EFFECTS, 'wages{age}_ein_fes'),
117
+ 'Figure 3: 2yr attendance at age 20': (EFFECTS, 'att_2yr_age20'),
118
+ 'Figure 3: 4yr attendance at age 20': (EFFECTS, 'att_4yr_age20'),
119
+ 'Figure 3: elite attendance at age 20': (EFFECTS, 'att_elite_age20'),
120
+ 'Figure 3: AA completion by age 24': (EFFECTS, 'cmp_AA_by_age24'),
121
+ 'Figure 3: BA completion by age 24': (EFFECTS, 'cmp_BA_by_age24'),
122
+ 'Figure 3: elite completion by age 24': (EFFECTS, 'cmp_elite_by_age24'),
123
+ 'Figure A3: network earnings': (NETWORKS, 'wages25'),
124
+ }
125
+ # COLUMN HEADERS: left = what the code needs; right = your Excel header.
126
+ # Change independently for each sheet. E.g. SUMMARY 'variable': 'depvar',
127
+ # 'mean': 'average', 'count': 'N_ppl'. Add any other field rename here.
128
+ columns = {
129
+ SUMMARY: {'sample': 'sample', 'variable': 'variable',
130
+ 'mean': 'mean', 'sd': 'sd', 'count': 'count'},
131
+ BALANCE: {'sample': 'sample', 'depvar': 'depvar', 'control_mean': 'control_mean',
132
+ 'coef': 'coef', 'se': 'se', 'N_ppl': 'N_ppl',
133
+ 'f_stat': 'f_stat', 'p_value': 'p_value'},
134
+ EFFECTS: {'sample': 'sample', 'depvar': 'depvar', 'spec': 'spec',
135
+ 'coef': 'coef', 'se': 'se', 'ccm': 'ccm', 'N_ppl': 'N_ppl'},
136
+ FIRST_STAGE: {'sample':'sample', 'depvar':'depvar', 'spec':'spec',
137
+ 'coef':'coef', 'se':'se', 'N_ppl':'N_ppl', 'control_mean':'control_mean'},
138
+ ITT: {'sample':'sample', 'depvar':'depvar', 'spec':'spec',
139
+ 'coef':'coef', 'se':'se', 'N_ppl':'N_ppl', 'control_mean':'control_mean'},
140
+ CONVERSIONS: {'depvar':'depvar', 'earnings_gain':'earnings_gain'},
141
+ NETWORKS: {'sample':'sample', 'depvar':'depvar', 'spec':'spec', 'coef':'coef', 'se':'se'},
142
+ OLS: {'sample': 'sample', 'depvar': 'depvar', 'spec': 'spec',
143
+ 'comparison_mean': 'comparison_mean', 'coef':'coef', 'se':'se', 'N_ppl':'N_ppl'},
144
+ PREDICTIONS: {'sample': 'sample', 'depvar': 'depvar', 'spec': 'spec', 'coef': 'coef'},
145
+ }
146
+ # SAMPLE VALUES: left = the table's sample; right = its name in your workbook.
147
+ # 'tract_sample' is the tract comparison sample; 'national' is the national sample.
148
+ # 'aligned_20plus' is the college sample aligned to 1098T coverage.
149
+ samples = {
150
+ 'figure3_attendance': 'overall_20plus',
151
+ 'figure3_completion': 'overall_25plus',
152
+ 'aligned_20plus': 'aligned_20plus',
153
+ 'black_20plus': 'black_20plus',
154
+ 'black_25plus': 'black_25plus',
155
+ 'entry_high_school_20plus': 'entry_high_school_20plus',
156
+ 'entry_high_school_25plus': 'entry_high_school_25plus',
157
+ 'entry_middle_school_20plus': 'entry_middle_school_20plus',
158
+ 'entry_middle_school_25plus': 'entry_middle_school_25plus',
159
+ 'ever_college_25plus': 'ever_college_25plus',
160
+ 'female_20plus': 'female_20plus',
161
+ 'female_25plus': 'female_25plus',
162
+ 'hispanic_20plus': 'hispanic_20plus',
163
+ 'hispanic_25plus': 'hispanic_25plus',
164
+ 'male_20plus': 'male_20plus',
165
+ 'male_25plus': 'male_25plus',
166
+ 'national': 'national',
167
+ 'neighborhood_income_high_20plus': 'neighborhood_income_high_20plus',
168
+ 'neighborhood_income_high_25plus': 'neighborhood_income_high_25plus',
169
+ 'neighborhood_income_low_20plus': 'neighborhood_income_low_20plus',
170
+ 'neighborhood_income_low_25plus': 'neighborhood_income_low_25plus',
171
+ 'never_college_25plus': 'never_college_25plus',
172
+ 'overall_20plus': 'overall_20plus',
173
+ 'overall_25plus': 'overall_25plus',
174
+ 'parent_income_high_20plus': 'parent_income_high_20plus',
175
+ 'parent_income_high_25plus': 'parent_income_high_25plus',
176
+ 'parent_income_low_20plus': 'parent_income_low_20plus',
177
+ 'parent_income_low_25plus': 'parent_income_low_25plus',
178
+ 'tract_sample': 'tract_sample',
179
+ }
180
+ # If an alternative result is identified by SAMPLE instead of SPEC, set it here.
181
+ # Example: {'Age 25 earnings': {'initial_offer': 'your_initial_offer_sample'}}
182
+ # This replaces that result's sample and removes its spec filter.
183
+ sample_instead_of_spec = {}
184
+ # Missing spec columns are automatic. Only rename existing spec values here.
185
+ specs = {} # Example: {'baseline': 'none'}
186
+ # Every result is new/red by default. To reuse one result in black, add:
187
+ # reuse_disclosed = {'ITT: Age 25 earnings': ('ITT', 'wages25')}
188
+ # Reused results keep the original column/sample/spec names. New-name maps
189
+ # below apply only to PENDING_INPUT, so they cannot alter old-workbook lookups.
190
+ reuse_disclosed = {}
191
+ # Summary means/SDs: 1 for percentage points (0-100), 100 for proportions (0-1).
192
+ summary_percent_scale = 1
193
+ # Figure A3: display label -> sample value in the network-results sheet.
194
+ # Replace this example roster with your actual networks; do not include Overall.
195
+ networks = {
196
+ 'Network A': 'network_a', 'Network B': 'network_b',
197
+ 'Network C': 'network_c', 'Network D': 'network_d',
198
+ 'Network E': 'network_e', 'Network F': 'network_f',
199
+ }
200
+ # External benchmarks used in the test-score mediation calculation.
201
+ benchmarks = {'credo_math_effect_per_year': 0.0974,
202
+ 'star_earnings_return_per_sd': 13.17, 'years_of_exposure': 3}
203
+ return resolve_inputs(inputs, columns=columns, samples=samples, specs=specs,
204
+ sample_instead_of_spec=sample_instead_of_spec,
205
+ reuse_disclosed=reuse_disclosed,
206
+ summary_percent_scale=summary_percent_scale,
207
+ networks=networks, benchmarks=benchmarks)
208
+
209
+
210
+ ###########################################################
211
+ # Build the approved exhibit packet with pdfLaTeX directly.
212
+ ###########################################################
213
+
214
+ def build_latex(mapping=None, disclosed_path=None, pending_path=None, output_path=None, build_dir=None):
215
+ mapping = input_mapping() if mapping is None else mapping
216
+ paths = {'disclosed': Path(disclosed_path or DISCLOSED_INPUT), 'pending': Path(pending_path or PENDING_INPUT)}
217
+ output_path = Path(output_path or OUTPUT_PDF)
218
+ build_dir = Path(build_dir or BUILD_DIR)
219
+ if build_dir.resolve() in {SOURCE_DIR.resolve(), PAPER_ROOT.resolve()}:
220
+ raise ValueError('BUILD_DIR must be separate from source files.')
221
+ bindings = json.loads(BINDINGS_FILE.read_text())
222
+ main, appendix, included, excluded, references = select_exhibits(SOURCE_DIR, EXCLUDE_TABLES, EXCLUDE_FIGURES)
223
+ data = SplitInputs(paths, mapping, color_pending=COLOR_PENDING)
224
+ table_names = {e['name'] for e in included if e['kind']=='table'}
225
+ # Validate before invoking TeX. Only selected exhibit inputs are required.
226
+ active_cells = {key:b for key,b in bindings['cells'].items() if key.split('.')[0] in table_names}
227
+ try:
228
+ for binding in active_cells.values(): data.render(binding)
229
+ build_dir.mkdir(parents=True, exist_ok=True)
230
+ shutil.copytree(SOURCE_DIR/'paper', build_dir/'paper', dirs_exist_ok=True)
231
+ # Write tables into a controlled build tree, never over inputs/templates.
232
+ build_tables(data, dict(bindings, cells=active_cells), SOURCE_DIR, build_dir)
233
+ figures = mapping['figures']
234
+ selected_figures = {e['name'] for e in included if e['kind']=='figure'}
235
+ required_figures = {key for key, exhibit in FIGURE_EXHIBITS.items() if exhibit in selected_figures}
236
+ if required_figures - figures.keys():
237
+ raise ValueError(f'Missing selected figure mappings: {sorted(required_figures - figures.keys())}')
238
+ figures = {k:v for k,v in figures.items() if FIGURE_EXHIBITS[k] in selected_figures}
239
+ figure_audit = make_figures(data, figures, build_dir/'outputs/figures')
240
+ # Table notes use the same live inputs as the corresponding numeric cells.
241
+ main = re.sub(r'@@([^@]+)@@', lambda m: data.render(active_cells[m[1]]), main)
242
+ appendix = re.sub(r'@@([^@]+)@@', lambda m: data.render(active_cells[m[1]]), appendix)
243
+ preview = any('preview' in str(row.get('source', '')).lower() or
244
+ 'synthetic' in str(row.get('source', '')).lower() or
245
+ 'hypothetical' in str(row.get('source', '')).lower()
246
+ for row in data.records.values())
247
+ (build_dir/'paper/main-tables.tex').write_text(main)
248
+ (build_dir/'paper/appendix.tex').write_text(appendix)
249
+ wrapper = r'''\documentclass[11pt,english]{article}
250
+ \makeatletter
251
+ \def\input@path{{paper/}}
252
+ \makeatother
253
+ \input{preamble}
254
+ \begin{document}
255
+ \input{main-tables}
256
+ \begin{appendices}
257
+ \input{appendix}
258
+ \end{appendices}
259
+ \end{document}
260
+ '''
261
+ (build_dir/'main.tex').write_text(wrapper)
262
+ engine = shutil.which(str(PDFLATEX))
263
+ if engine is None:
264
+ raise RuntimeError('pdfLaTeX was not found. Set PDFLATEX in paths.py to its executable path.')
265
+ command = [engine, '-interaction=nonstopmode', '-halt-on-error', '-file-line-error', '-no-shell-escape', 'main.tex']
266
+ with (build_dir/'compile-output.txt').open('w') as log:
267
+ for pass_number in range(1,4):
268
+ result = subprocess.run(command, cwd=build_dir, capture_output=True, text=True)
269
+ log.write(f'PASS {pass_number}\n'+result.stdout+'\n'+result.stderr)
270
+ if result.returncode:
271
+ raise RuntimeError('pdfLaTeX failed. See '+str(build_dir/'compile-output.txt')+'\n'+(result.stdout+result.stderr)[-3500:])
272
+ pdf = PdfReader(build_dir/'main.pdf')
273
+ if len(pdf.pages) != len(included):
274
+ raise ValueError(f'Expected {len(included)} exhibit pages; got {len(pdf.pages)}.')
275
+ for page, exhibit in zip(pdf.pages, included):
276
+ if exhibit['title'].upper() not in page.extract_text():
277
+ raise ValueError('Missing or renumbered exhibit: '+exhibit['title'])
278
+ output_path.parent.mkdir(parents=True, exist_ok=True)
279
+ shutil.copyfile(build_dir/'main.pdf', output_path)
280
+ audit = {'engine':engine, 'pages':len(pdf.pages), 'included':[e['title'] for e in included],
281
+ 'preview':preview,
282
+ 'input_sha256':{role:hashlib.sha256(paths[role].read_bytes()).hexdigest() for role in data.books},
283
+ 'mapping':mapping, 'used':data.used, 'figures':figure_audit,
284
+ 'cells':{key:{'value':data.value(b), 'pending':data.is_pending(b)} for key,b in active_cells.items()}}
285
+ output_path.with_suffix('.audit.json').write_text(json.dumps(audit,indent=2)+'\n')
286
+ print(f'Created {output_path}: {len(pdf.pages)} pages with pdfLaTeX')
287
+ return audit
288
+ finally:
289
+ data.close()
290
+
291
+
292
+ if __name__ == '__main__':
293
+ build_latex()
@@ -0,0 +1,52 @@
1
+ # --- Import necessary packages ---
2
+ import re
3
+
4
+ ###########################################################
5
+ # Select complete exhibits and retain their manuscript numbering
6
+ ###########################################################
7
+
8
+ FIGURES = (
9
+ 'appendix-credo-distributions',
10
+ 'appendix-college-by-age-placeholder',
11
+ 'appendix-cmo-leave-one-out',
12
+ 'appendix-earnings-by-age-firm-chars',
13
+ )
14
+ FLOAT = re.compile(r'\\begin\{(table|figure)\}.*?\\end\{\1\}',re.S)
15
+
16
+
17
+ def select_exhibits(source_root, exclude_tables, exclude_figures):
18
+ main=(source_root/'paper/tables-and-figures.tex').read_text()
19
+ # Retain the main figures; the catalog owns exclusions and manuscript numbering.
20
+ appendix=(source_root/'paper/appendix.tex').read_text()
21
+ catalog=[]
22
+ for section,text in [('main',main),('appendix',appendix)]:
23
+ counters={'table':0,'figure':0}
24
+ for match in FLOAT.finditer(text):
25
+ kind=match[1];counters[kind]+=1;block=match[0]
26
+ label=re.search(r'\\label\{([^}]+)\}',block)[1]
27
+ name=re.search(r'\\input\{[^}]*?/([^/}]+)\}',block)[1] if kind=='table' else label.removeprefix('fig:')
28
+ number=('A.' if section=='appendix' else '')+str(counters[kind])
29
+ catalog.append(dict(section=section,kind=kind,name=name,label=label,number=number,
30
+ title=kind.title()+' '+number,counter=counters[kind]-1))
31
+ excluded=set()
32
+ for kind,requested in [('table',exclude_tables),('figure',exclude_figures)]:
33
+ if isinstance(requested,str):raise ValueError(f'EXCLUDE_{kind.upper()}S must be a list, not a string')
34
+ aliases={alias:e['name'] for e in catalog if e['kind']==kind for alias in [e['name'],e['title']]}
35
+ for name in requested:
36
+ if name not in aliases:raise ValueError(f'Unknown excluded {kind}: {name!r}. Choose from: {sorted(aliases)}')
37
+ excluded.add(aliases[name])
38
+ retained=[e for e in catalog if e['name'] not in excluded]
39
+ if not retained:raise ValueError('All exhibits are excluded. Include at least one table or figure.')
40
+ if excluded:
41
+ for section,text in [('main',main),('appendix',appendix)]:
42
+ entries=iter(e for e in catalog if e['section']==section)
43
+ def replace(match):
44
+ entry=next(entries)
45
+ if entry['name'] in excluded:return ''
46
+ return '\\setcounter{'+entry['kind']+'}{'+str(entry['counter'])+'}\n'+match[0]
47
+ filtered=FLOAT.sub(replace,text)
48
+ if section=='main':main=filtered
49
+ else:appendix=filtered
50
+ references='\n'.join('\\newlabel{'+e['label']+'}{{'+e['number']+'}{}{}{'+e['kind']+'.'+e['number']+'}{}}%'
51
+ for e in catalog if e['name'] in excluded)
52
+ return main,appendix,retained,[e for e in catalog if e['name'] in excluded],references
@@ -0,0 +1,51 @@
1
+ """Dependency-free extended Wilkinson breaks, as used by scales::extended_breaks.
2
+
3
+ Adapted from mizani.breaks.breaks_extended (Hassan Kibirige / Justin Talbot).
4
+ See TICK_ALGORITHM_LICENSE.txt. Kept local to avoid numpy/pandas dependencies
5
+ in the portable workbook-to-LaTeX builder.
6
+ """
7
+ import math
8
+
9
+
10
+ def extended_breaks(lo, hi):
11
+ if hi == lo:
12
+ return [lo]
13
+ span = hi-lo
14
+ best_score, best = -2, []
15
+ q_values = [1, 5, 2, 2.5, 4, 3]
16
+ j = 1
17
+ while True:
18
+ for index, q in enumerate(q_values):
19
+ sm = (5-index)/5 + 1-j
20
+ if .25*sm + .75 < best_score:
21
+ return [v for v in best if lo-1e-9 <= v <= hi+1e-9]
22
+ k = 2
23
+ while True:
24
+ dm = 2-(k-1)/4 if k >= 5 else 1
25
+ if .25*sm + .2 + .5*dm + .05 < best_score:
26
+ break
27
+ z = math.ceil(math.log10(span/(k+1)/j/q))
28
+ while True:
29
+ step = j*q*10**z
30
+ cm = 1-((step*(k-1)-span)/2)**2/(.1*span)**2 if step*(k-1)>span else 1
31
+ if .25*sm + .2*cm + .5*dm + .05 < best_score:
32
+ break
33
+ first = math.floor(hi/step)*j-(k-1)*j
34
+ last = math.ceil(lo/step)*j
35
+ if first > last:
36
+ break
37
+ for start in range(first,last+1):
38
+ a = start*step/j
39
+ b = a+step*(k-1)
40
+ zero = int((a % step < 1e-10 or step-a % step < 1e-10) and a<=0<=b)
41
+ simplicity = (5-index)/5 + zero-j
42
+ coverage = 1-.5*((hi-b)**2+(lo-a)**2)/(.1*span)**2
43
+ density_ratio = ((k-1)/(b-a))/(4/(max(b,hi)-min(a,lo)))
44
+ density = 2-max(density_ratio,1/density_ratio)
45
+ score = .25*simplicity+.2*coverage+.5*density+.05
46
+ if score > best_score:
47
+ best_score = score
48
+ best = [a+i*step for i in range(k)]
49
+ z += 1
50
+ k += 1
51
+ j += 1
Binary file
@@ -0,0 +1,96 @@
1
+ Fonts are (c) Bitstream (see below). DejaVu changes are in public domain. Glyphs imported from Arev fonts are (c) Tavmjung Bah (see below)
2
+
3
+ Bitstream Vera Fonts Copyright
4
+ ------------------------------
5
+
6
+ Copyright (c) 2003 by Bitstream, Inc. All Rights Reserved. Bitstream Vera is
7
+ a trademark of Bitstream, Inc.
8
+
9
+ Permission is hereby granted, free of charge, to any person obtaining a copy
10
+ of the fonts accompanying this license ("Fonts") and associated
11
+ documentation files (the "Font Software"), to reproduce and distribute the
12
+ Font Software, including without limitation the rights to use, copy, merge,
13
+ publish, distribute, and/or sell copies of the Font Software, and to permit
14
+ persons to whom the Font Software is furnished to do so, subject to the
15
+ following conditions:
16
+
17
+ The above copyright and trademark notices and this permission notice shall
18
+ be included in all copies of one or more of the Font Software typefaces.
19
+
20
+ The Font Software may be modified, altered, or added to, and in particular
21
+ the designs of glyphs or characters in the Fonts may be modified and
22
+ additional glyphs or characters may be added to the Fonts, only if the fonts
23
+ are renamed to names not containing either the words "Bitstream" or the word
24
+ "Vera".
25
+
26
+ This License becomes null and void to the extent applicable to Fonts or Font
27
+ Software that has been modified and is distributed under the "Bitstream
28
+ Vera" names.
29
+
30
+ The Font Software may be sold as part of a larger software package but no
31
+ copy of one or more of the Font Software typefaces may be sold by itself.
32
+
33
+ THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
34
+ OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF MERCHANTABILITY,
35
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OF COPYRIGHT, PATENT,
36
+ TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL BITSTREAM OR THE GNOME
37
+ FOUNDATION BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, INCLUDING
38
+ ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL DAMAGES,
39
+ WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF
40
+ THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM OTHER DEALINGS IN THE
41
+ FONT SOFTWARE.
42
+
43
+ Except as contained in this notice, the names of Gnome, the Gnome
44
+ Foundation, and Bitstream Inc., shall not be used in advertising or
45
+ otherwise to promote the sale, use or other dealings in this Font Software
46
+ without prior written authorization from the Gnome Foundation or Bitstream
47
+ Inc., respectively. For further information, contact: fonts at gnome dot
48
+ org.
49
+
50
+ Arev Fonts Copyright
51
+ ------------------------------
52
+
53
+ Copyright (c) 2006 by Tavmjong Bah. All Rights Reserved.
54
+
55
+ Permission is hereby granted, free of charge, to any person obtaining
56
+ a copy of the fonts accompanying this license ("Fonts") and
57
+ associated documentation files (the "Font Software"), to reproduce
58
+ and distribute the modifications to the Bitstream Vera Font Software,
59
+ including without limitation the rights to use, copy, merge, publish,
60
+ distribute, and/or sell copies of the Font Software, and to permit
61
+ persons to whom the Font Software is furnished to do so, subject to
62
+ the following conditions:
63
+
64
+ The above copyright and trademark notices and this permission notice
65
+ shall be included in all copies of one or more of the Font Software
66
+ typefaces.
67
+
68
+ The Font Software may be modified, altered, or added to, and in
69
+ particular the designs of glyphs or characters in the Fonts may be
70
+ modified and additional glyphs or characters may be added to the
71
+ Fonts, only if the fonts are renamed to names not containing either
72
+ the words "Tavmjong Bah" or the word "Arev".
73
+
74
+ This License becomes null and void to the extent applicable to Fonts
75
+ or Font Software that has been modified and is distributed under the
76
+ "Tavmjong Bah Arev" names.
77
+
78
+ The Font Software may be sold as part of a larger software package but
79
+ no copy of one or more of the Font Software typefaces may be sold by
80
+ itself.
81
+
82
+ THE FONT SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
83
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTIES OF
84
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT
85
+ OF COPYRIGHT, PATENT, TRADEMARK, OR OTHER RIGHT. IN NO EVENT SHALL
86
+ TAVMJONG BAH BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
87
+ INCLUDING ANY GENERAL, SPECIAL, INDIRECT, INCIDENTAL, OR CONSEQUENTIAL
88
+ DAMAGES, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
89
+ FROM, OUT OF THE USE OR INABILITY TO USE THE FONT SOFTWARE OR FROM
90
+ OTHER DEALINGS IN THE FONT SOFTWARE.
91
+
92
+ Except as contained in this notice, the name of Tavmjong Bah shall not
93
+ be used in advertising or otherwise to promote the sale, use or other
94
+ dealings in this Font Software without prior written authorization
95
+ from Tavmjong Bah. For further information, contact: tavmjong @ free
96
+ . fr.