cvprofiles 2.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- cvprofiles-2.0.0/.env.example +10 -0
- cvprofiles-2.0.0/.github/workflows/ci.yml +100 -0
- cvprofiles-2.0.0/.gitignore +74 -0
- cvprofiles-2.0.0/AGENTS.md +112 -0
- cvprofiles-2.0.0/CHANGELOG.md +97 -0
- cvprofiles-2.0.0/LICENSE +21 -0
- cvprofiles-2.0.0/PKG-INFO +150 -0
- cvprofiles-2.0.0/README.md +116 -0
- cvprofiles-2.0.0/data/fixtures/README.md +9 -0
- cvprofiles-2.0.0/data/fixtures/mean_order_v1/beta.yaml +4 -0
- cvprofiles-2.0.0/data/fixtures/mean_order_v1/network.yaml +10 -0
- cvprofiles-2.0.0/data/fixtures/mean_order_v1/roles.json +7 -0
- cvprofiles-2.0.0/data/fixtures/mean_order_v1/scores.csv +11 -0
- cvprofiles-2.0.0/data/fixtures/mini_v1/README.md +29 -0
- cvprofiles-2.0.0/data/fixtures/mini_v1/anchors.yaml +12 -0
- cvprofiles-2.0.0/data/fixtures/mini_v1/beta.yaml +4 -0
- cvprofiles-2.0.0/data/fixtures/mini_v1/expected_freeze.json +34 -0
- cvprofiles-2.0.0/data/fixtures/mini_v1/network.yaml +15 -0
- cvprofiles-2.0.0/data/fixtures/mini_v1/network_harsh.yaml +16 -0
- cvprofiles-2.0.0/data/fixtures/mini_v1/roles.json +7 -0
- cvprofiles-2.0.0/data/fixtures/mini_v1/scores.csv +11 -0
- cvprofiles-2.0.0/data/fixtures/ols_v1/beta.yaml +5 -0
- cvprofiles-2.0.0/data/fixtures/ols_v1/network.yaml +10 -0
- cvprofiles-2.0.0/data/fixtures/ols_v1/roles.json +7 -0
- cvprofiles-2.0.0/data/fixtures/ols_v1/scores.csv +11 -0
- cvprofiles-2.0.0/data/fixtures/rank_agree_v1/beta.yaml +4 -0
- cvprofiles-2.0.0/data/fixtures/rank_agree_v1/network.yaml +9 -0
- cvprofiles-2.0.0/data/fixtures/rank_agree_v1/roles.json +7 -0
- cvprofiles-2.0.0/data/fixtures/rank_agree_v1/scores.csv +11 -0
- cvprofiles-2.0.0/data/synthetic/README.md +1 -0
- cvprofiles-2.0.0/docs/01_Project_Overview.md +78 -0
- cvprofiles-2.0.0/docs/02_System_Architecture.md +156 -0
- cvprofiles-2.0.0/docs/03_Methodology.md +145 -0
- cvprofiles-2.0.0/docs/04_Synthetic_DGPs.md +112 -0
- cvprofiles-2.0.0/docs/05_Pre_Registration.md +130 -0
- cvprofiles-2.0.0/docs/06_Tech_Stack.md +86 -0
- cvprofiles-2.0.0/docs/07_Software_Development_Strategy.md +97 -0
- cvprofiles-2.0.0/docs/08_Observability_and_Evaluations.md +97 -0
- cvprofiles-2.0.0/docs/09_MVP_Plan.md +134 -0
- cvprofiles-2.0.0/docs/10_Open_Questions.md +69 -0
- cvprofiles-2.0.0/docs/11_Glossary.md +28 -0
- cvprofiles-2.0.0/docs/12_Decision_Engineering_Log.md +809 -0
- cvprofiles-2.0.0/docs/13_Evaluations_Log.md +484 -0
- cvprofiles-2.0.0/docs/14_Researcher_Input_Guide.md +367 -0
- cvprofiles-2.0.0/docs/15_MVP_Release_Checklist.md +72 -0
- cvprofiles-2.0.0/docs/16_Paper_Protocol_Freeze.md +157 -0
- cvprofiles-2.0.0/docs/17_H5_Trust_Design.md +125 -0
- cvprofiles-2.0.0/docs/18_Measure_Discipline_Plan.md +118 -0
- cvprofiles-2.0.0/docs/PROJECT_MANIFEST.md +126 -0
- cvprofiles-2.0.0/docs/README.md +18 -0
- cvprofiles-2.0.0/evals/h5_trust/README.md +79 -0
- cvprofiles-2.0.0/evals/h5_trust/build_dataset.py +471 -0
- cvprofiles-2.0.0/evals/h5_trust/data/anchors_h5_trust.yaml +19 -0
- cvprofiles-2.0.0/evals/h5_trust/data/beta_h5_trust.yaml +4 -0
- cvprofiles-2.0.0/evals/h5_trust/data/network_h5_trust.yaml +20 -0
- cvprofiles-2.0.0/evals/h5_trust/data/roles_h5_trust.json +23 -0
- cvprofiles-2.0.0/evals/h5_trust/data/score_manifest.json +40 -0
- cvprofiles-2.0.0/evals/h5_trust/data/scores.csv +36 -0
- cvprofiles-2.0.0/evals/h5_trust/proof_summary.json +58 -0
- cvprofiles-2.0.0/evals/h5_trust/verify_audit.py +149 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/README.md +86 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/build_dataset.py +217 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/data/beta.yaml +4 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/data/network_harsh.yaml +9 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/data/network_oracle.yaml +15 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/data/roles.json +18 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/data/roles_skew.json +16 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/data/scores.csv +20641 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/data/scores_skew.csv +20641 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/proof_summary.json +177 -0
- cvprofiles-2.0.0/evals/realworld/calhousing_validity/verify_audit.py +314 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/README.md +63 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/build_dataset.py +191 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/data/beta.yaml +4 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/data/network_harsh.yaml +9 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/data/network_oracle.yaml +15 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/data/roles.json +17 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/data/scores.csv +2193 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/proof_summary.json +88 -0
- cvprofiles-2.0.0/evals/realworld/spam_validity/verify_audit.py +198 -0
- cvprofiles-2.0.0/evals/synthetic/README.md +44 -0
- cvprofiles-2.0.0/evals/synthetic/v0_poc.py +720 -0
- cvprofiles-2.0.0/pyproject.toml +78 -0
- cvprofiles-2.0.0/reports/.gitkeep +0 -0
- cvprofiles-2.0.0/reports/runs/.gitkeep +0 -0
- cvprofiles-2.0.0/reports/summaries/.gitkeep +0 -0
- cvprofiles-2.0.0/reports/summaries/README.md +13 -0
- cvprofiles-2.0.0/reports/summaries/h5_trust_evidence_summary.json +139 -0
- cvprofiles-2.0.0/reports/summaries/v0_1_poc_summary.json +76 -0
- cvprofiles-2.0.0/reports/summaries/v1_0_package_synth_summary.json +563 -0
- cvprofiles-2.0.0/reports/summaries/v1_1_package_synth_summary.json +1050 -0
- cvprofiles-2.0.0/reports/summaries/v1_1_protocol_synth_mc50_summary.json +5196 -0
- cvprofiles-2.0.0/src/cvprofiles/__init__.py +6 -0
- cvprofiles-2.0.0/src/cvprofiles/__main__.py +5 -0
- cvprofiles-2.0.0/src/cvprofiles/anchors/__init__.py +27 -0
- cvprofiles-2.0.0/src/cvprofiles/anchors/pipeline.py +136 -0
- cvprofiles-2.0.0/src/cvprofiles/cli.py +213 -0
- cvprofiles-2.0.0/src/cvprofiles/freeze.py +279 -0
- cvprofiles-2.0.0/src/cvprofiles/identify/__init__.py +17 -0
- cvprofiles-2.0.0/src/cvprofiles/identify/beta_fn.py +77 -0
- cvprofiles-2.0.0/src/cvprofiles/identify/pipeline.py +195 -0
- cvprofiles-2.0.0/src/cvprofiles/identify/slacks.py +129 -0
- cvprofiles-2.0.0/src/cvprofiles/inference/__init__.py +29 -0
- cvprofiles-2.0.0/src/cvprofiles/inference/bootstrap.py +174 -0
- cvprofiles-2.0.0/src/cvprofiles/inference/delta_grid.py +163 -0
- cvprofiles-2.0.0/src/cvprofiles/inference/theta_grid.py +182 -0
- cvprofiles-2.0.0/src/cvprofiles/pipeline.py +321 -0
- cvprofiles-2.0.0/src/cvprofiles/report/__init__.py +7 -0
- cvprofiles-2.0.0/src/cvprofiles/report/pipeline.py +159 -0
- cvprofiles-2.0.0/src/cvprofiles/report/templates/report.html.j2 +385 -0
- cvprofiles-2.0.0/src/cvprofiles/restrict/__init__.py +21 -0
- cvprofiles-2.0.0/src/cvprofiles/restrict/pipeline.py +208 -0
- cvprofiles-2.0.0/src/cvprofiles/schemas/__init__.py +18 -0
- cvprofiles-2.0.0/src/cvprofiles/schemas/beta.py +31 -0
- cvprofiles-2.0.0/src/cvprofiles/schemas/network.py +98 -0
- cvprofiles-2.0.0/src/cvprofiles/schemas/run.py +56 -0
- cvprofiles-2.0.0/src/cvprofiles/schemas/scores.py +84 -0
- cvprofiles-2.0.0/src/cvprofiles/score/__init__.py +7 -0
- cvprofiles-2.0.0/src/cvprofiles/score/pipeline.py +197 -0
- cvprofiles-2.0.0/src/cvprofiles/synth/__init__.py +32 -0
- cvprofiles-2.0.0/src/cvprofiles/synth/battery.py +323 -0
- cvprofiles-2.0.0/src/cvprofiles/synth/dgp.py +153 -0
- cvprofiles-2.0.0/src/cvprofiles/synth/metrics.py +147 -0
- cvprofiles-2.0.0/src/cvprofiles/synth/oracle_r.py +61 -0
- cvprofiles-2.0.0/tests/conftest.py +63 -0
- cvprofiles-2.0.0/tests/test_anchors.py +181 -0
- cvprofiles-2.0.0/tests/test_bootstrap.py +205 -0
- cvprofiles-2.0.0/tests/test_delta_grid.py +147 -0
- cvprofiles-2.0.0/tests/test_freeze.py +256 -0
- cvprofiles-2.0.0/tests/test_h5_trust_build.py +357 -0
- cvprofiles-2.0.0/tests/test_h5_trust_engine_smoke.py +111 -0
- cvprofiles-2.0.0/tests/test_h5_trust_verify.py +272 -0
- cvprofiles-2.0.0/tests/test_identify.py +193 -0
- cvprofiles-2.0.0/tests/test_import_graph.py +127 -0
- cvprofiles-2.0.0/tests/test_mean_order.py +133 -0
- cvprofiles-2.0.0/tests/test_mini_fixture_contract.py +91 -0
- cvprofiles-2.0.0/tests/test_ols_coef.py +170 -0
- cvprofiles-2.0.0/tests/test_pipeline_e2e.py +168 -0
- cvprofiles-2.0.0/tests/test_rank_agree.py +130 -0
- cvprofiles-2.0.0/tests/test_report.py +119 -0
- cvprofiles-2.0.0/tests/test_restrict.py +93 -0
- cvprofiles-2.0.0/tests/test_schemas_beta.py +36 -0
- cvprofiles-2.0.0/tests/test_schemas_network.py +132 -0
- cvprofiles-2.0.0/tests/test_schemas_scores.py +67 -0
- cvprofiles-2.0.0/tests/test_score.py +111 -0
- cvprofiles-2.0.0/tests/test_synth_battery.py +148 -0
- cvprofiles-2.0.0/tests/test_theta_grid.py +178 -0
- cvprofiles-2.0.0/tests/test_v11_protocol_synth_mc50_audit.py +145 -0
- cvprofiles-2.0.0/tests/test_v11_wiring.py +228 -0
- cvprofiles-2.0.0/tests/test_v2_wiring.py +307 -0
- cvprofiles-2.0.0/tools/make_h5_trust_summary.py +111 -0
- cvprofiles-2.0.0/tools/refresh_mini_golden.py +92 -0
- cvprofiles-2.0.0/tools/v11_protocol_synth_mc50.py +266 -0
- cvprofiles-2.0.0/tools/v11_synth_summary.py +303 -0
- cvprofiles-2.0.0/tools/verify_h5_trust.py +210 -0
- cvprofiles-2.0.0/tools/verify_v11_protocol_synth_mc50.py +371 -0
- cvprofiles-2.0.0/tutorials/README.md +41 -0
- cvprofiles-2.0.0/tutorials/cvprofiles_tutorial.ipynb +255 -0
- cvprofiles-2.0.0/uv.lock +1122 -0
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# cvprofiles environment variables
|
|
2
|
+
# Core engine needs NO secrets. Frozen scores + YAML configs suffice.
|
|
3
|
+
#
|
|
4
|
+
# Optional upstream *scoring recipes* (outside the engine) may need keys later.
|
|
5
|
+
# Never import optional scorer env vars from identify/report code paths.
|
|
6
|
+
|
|
7
|
+
# Example placeholders only — not used by MVP engine:
|
|
8
|
+
# OPENAI_API_KEY=
|
|
9
|
+
# ANTHROPIC_API_KEY=
|
|
10
|
+
# HF_TOKEN=
|
|
@@ -0,0 +1,100 @@
|
|
|
1
|
+
name: ci
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main, "feat/**"]
|
|
6
|
+
pull_request:
|
|
7
|
+
branches: [main]
|
|
8
|
+
|
|
9
|
+
concurrency:
|
|
10
|
+
group: ci-${{ github.workflow }}-${{ github.ref }}
|
|
11
|
+
cancel-in-progress: true
|
|
12
|
+
|
|
13
|
+
jobs:
|
|
14
|
+
test:
|
|
15
|
+
name: test (py${{ matrix.python-version }})
|
|
16
|
+
runs-on: ubuntu-latest
|
|
17
|
+
strategy:
|
|
18
|
+
fail-fast: false
|
|
19
|
+
matrix:
|
|
20
|
+
python-version: ["3.11", "3.12"]
|
|
21
|
+
|
|
22
|
+
steps:
|
|
23
|
+
- name: Checkout
|
|
24
|
+
uses: actions/checkout@v4
|
|
25
|
+
|
|
26
|
+
- name: Install uv
|
|
27
|
+
uses: astral-sh/setup-uv@v5
|
|
28
|
+
with:
|
|
29
|
+
enable-cache: true
|
|
30
|
+
python-version: ${{ matrix.python-version }}
|
|
31
|
+
|
|
32
|
+
- name: Sync project (dev extras)
|
|
33
|
+
run: uv sync --extra dev
|
|
34
|
+
|
|
35
|
+
- name: Ruff (src + tests + tools)
|
|
36
|
+
run: uv run ruff check src tests tools
|
|
37
|
+
|
|
38
|
+
- name: Mypy (strict package check)
|
|
39
|
+
run: uv run mypy src
|
|
40
|
+
|
|
41
|
+
- name: Pytest
|
|
42
|
+
run: uv run pytest -q --tb=short
|
|
43
|
+
|
|
44
|
+
- name: CLI version smoke
|
|
45
|
+
run: |
|
|
46
|
+
uv run cvprofiles --version
|
|
47
|
+
uv run python -c "import cvprofiles; assert cvprofiles.__version__ == '1.1.0a1'"
|
|
48
|
+
|
|
49
|
+
- name: Editable install graph hygiene (no LLM / no museum import)
|
|
50
|
+
run: |
|
|
51
|
+
uv run python - <<'PY'
|
|
52
|
+
import ast
|
|
53
|
+
from pathlib import Path
|
|
54
|
+
forbidden = ("openai", "anthropic", "httpx", "requests", "litellm")
|
|
55
|
+
museum = ("v0_poc", "evals.synthetic")
|
|
56
|
+
bad = []
|
|
57
|
+
for p in Path("src/cvprofiles").rglob("*.py"):
|
|
58
|
+
tree = ast.parse(p.read_text(), filename=str(p))
|
|
59
|
+
for n in ast.walk(tree):
|
|
60
|
+
if isinstance(n, ast.Import):
|
|
61
|
+
for a in n.names:
|
|
62
|
+
if any(a.name == f or a.name.startswith(f + ".") for f in forbidden):
|
|
63
|
+
bad.append(f"{p}:{a.name}")
|
|
64
|
+
if any(m in a.name for m in museum):
|
|
65
|
+
bad.append(f"{p}:museum:{a.name}")
|
|
66
|
+
elif isinstance(n, ast.ImportFrom):
|
|
67
|
+
mod = n.module or ""
|
|
68
|
+
if any(mod == f or mod.startswith(f + ".") for f in forbidden):
|
|
69
|
+
bad.append(f"{p}:{mod}")
|
|
70
|
+
if any(m in mod for m in museum):
|
|
71
|
+
bad.append(f"{p}:museum:{mod}")
|
|
72
|
+
assert not bad, bad
|
|
73
|
+
assert Path("evals/synthetic/v0_poc.py").is_file()
|
|
74
|
+
print("import-graph ok; museum present")
|
|
75
|
+
PY
|
|
76
|
+
|
|
77
|
+
- name: Mini fixture SCORE→REPORT smoke
|
|
78
|
+
run: |
|
|
79
|
+
uv run cvprofiles run \
|
|
80
|
+
--scores data/fixtures/mini_v1/scores.csv \
|
|
81
|
+
--roles data/fixtures/mini_v1/roles.json \
|
|
82
|
+
--network data/fixtures/mini_v1/network.yaml \
|
|
83
|
+
--beta data/fixtures/mini_v1/beta.yaml \
|
|
84
|
+
--out /tmp/cvprofiles_ci_mini \
|
|
85
|
+
--seed 0
|
|
86
|
+
test -f /tmp/cvprofiles_ci_mini/report.html
|
|
87
|
+
test -f /tmp/cvprofiles_ci_mini/range.json
|
|
88
|
+
uv run python - <<'PY'
|
|
89
|
+
import json
|
|
90
|
+
from pathlib import Path
|
|
91
|
+
r = json.loads(Path("/tmp/cvprofiles_ci_mini/range.json").read_text())
|
|
92
|
+
a = json.loads(Path("/tmp/cvprofiles_ci_mini/admissible.json").read_text())
|
|
93
|
+
assert r["empty"] is False
|
|
94
|
+
assert "m_slop" not in a["M_star"]
|
|
95
|
+
assert set(a["M_star"]) == {"m_good", "m_weak"}
|
|
96
|
+
print("mini fixture smoke ok", a["M_star"], r["L"], r["U"])
|
|
97
|
+
PY
|
|
98
|
+
|
|
99
|
+
# Optional: only runs if sklearn available path wanted later; keep default CI lean.
|
|
100
|
+
# Realworld spam audit remains a branch eval (needs sklearn + 20newsgroups cache).
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Byte-compiled / cache
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*$py.class
|
|
5
|
+
*.so
|
|
6
|
+
.Python
|
|
7
|
+
.mypy_cache/
|
|
8
|
+
.ruff_cache/
|
|
9
|
+
.pytest_cache/
|
|
10
|
+
.coverage
|
|
11
|
+
htmlcov/
|
|
12
|
+
.dmypy.json
|
|
13
|
+
dmypy.json
|
|
14
|
+
|
|
15
|
+
# Virtual envs
|
|
16
|
+
.venv/
|
|
17
|
+
venv/
|
|
18
|
+
.env
|
|
19
|
+
.env.*
|
|
20
|
+
!.env.example
|
|
21
|
+
|
|
22
|
+
# Packaging / build
|
|
23
|
+
build/
|
|
24
|
+
dist/
|
|
25
|
+
*.egg-info/
|
|
26
|
+
.eggs/
|
|
27
|
+
pip-wheel-metadata/
|
|
28
|
+
|
|
29
|
+
# Jupyter
|
|
30
|
+
.ipynb_checkpoints/
|
|
31
|
+
|
|
32
|
+
# OS / editor
|
|
33
|
+
.DS_Store
|
|
34
|
+
.idea/
|
|
35
|
+
.vscode/
|
|
36
|
+
*.swp
|
|
37
|
+
*~
|
|
38
|
+
|
|
39
|
+
# Project outputs (keep directory via .gitkeep)
|
|
40
|
+
# Per-seed run dumps are local; proof summaries may be committed.
|
|
41
|
+
reports/runs/*
|
|
42
|
+
!reports/runs/.gitkeep
|
|
43
|
+
# Ignore bulk under summaries except proof JSON + README
|
|
44
|
+
reports/summaries/*
|
|
45
|
+
!reports/summaries/.gitkeep
|
|
46
|
+
!reports/summaries/README.md
|
|
47
|
+
!reports/summaries/v0_1_poc_summary.json
|
|
48
|
+
!reports/summaries/v1_0_package_synth_summary.json
|
|
49
|
+
!reports/summaries/v1_1_package_synth_summary.json
|
|
50
|
+
!reports/summaries/v1_1_protocol_synth_mc50_summary.json
|
|
51
|
+
!reports/summaries/h5_trust_evidence_summary.json
|
|
52
|
+
!reports/.gitkeep
|
|
53
|
+
|
|
54
|
+
# Large local data (fixtures small enough to commit are OK under data/fixtures)
|
|
55
|
+
data/synthetic/generated/
|
|
56
|
+
*.parquet
|
|
57
|
+
!data/fixtures/**/*.parquet
|
|
58
|
+
|
|
59
|
+
# Secrets / local overrides
|
|
60
|
+
*.pem
|
|
61
|
+
credentials.json
|
|
62
|
+
|
|
63
|
+
# Hermes / agent local
|
|
64
|
+
.hermes/plans/
|
|
65
|
+
|
|
66
|
+
# Real-world audit run dumps (keep data/ + proof JSON; regenerate runs)
|
|
67
|
+
evals/realworld/**/runs_*/
|
|
68
|
+
evals/realworld/**/*.parquet
|
|
69
|
+
|
|
70
|
+
# H5 Trust: large local raw + aux caches (fetched/copied locally; frozen
|
|
71
|
+
# scores.csv + manifest are committed under evals/h5_trust/data/)
|
|
72
|
+
data/h5_trust_raw/
|
|
73
|
+
data/h5_trust_aux/
|
|
74
|
+
evals/h5_trust/runs_verify/
|
|
@@ -0,0 +1,112 @@
|
|
|
1
|
+
# cvprofiles — Agent Handoff Contract
|
|
2
|
+
|
|
3
|
+
## Mission
|
|
4
|
+
|
|
5
|
+
`cvprofiles` is an open academic methods package for **construct-validity profiles**: partial identification over a finite, researcher-supplied menu of measurement functions. It is not a scorer product, an LLM application, an automated theory-authoring system, or a generic causal-sensitivity package.
|
|
6
|
+
|
|
7
|
+
The package state machine is:
|
|
8
|
+
|
|
9
|
+
```text
|
|
10
|
+
SCORE → RESTRICT → IDENTIFY → REPORT
|
|
11
|
+
```
|
|
12
|
+
|
|
13
|
+
- **SCORE:** ingest and validate user-supplied unit×measure scores. Do not generate measures.
|
|
14
|
+
- **RESTRICT:** bind a researcher-authored nomological network and target functional.
|
|
15
|
+
- **IDENTIFY:** compute slacks, the admissible set `M*`, and the headline range `[L,U]`.
|
|
16
|
+
- **REPORT:** produce JSON/HTML audit artifacts a non-coder can inspect.
|
|
17
|
+
|
|
18
|
+
## Non-negotiable scientific and package locks
|
|
19
|
+
|
|
20
|
+
- The engine is **score-agnostic and model-free**. No LLM client belongs in the package/import graph.
|
|
21
|
+
- The menu is finite and researcher supplied. Prompt search and measurement generation are upstream workflows, not engine features.
|
|
22
|
+
- The headline range is the image of beta on **survivors only**: `[L,U] = [min B*, max B*]`. Rejected measures may be diagnostic but never enter the range.
|
|
23
|
+
- Empty `M*` and wide ranges are valid scientific outputs. Never automatically loosen thresholds to avoid an empty set.
|
|
24
|
+
- Bootstrap and theta-grid are additive diagnostics. The headline range remains unchanged; theta-grid settings are excluded from the freeze preimage.
|
|
25
|
+
- Paper-relevant numbers require frozen scores, pinned network/beta, fixed seed, package version, and the locked freeze/run-id contract.
|
|
26
|
+
- The annotated `v0.1` tag is immovable and must continue to peel to `fb62b48bcb704f60eee7d6641ed0a344eb72bfda`.
|
|
27
|
+
- `evals/synthetic/v0_poc.py` is a historical museum artifact. Keep it present and **unimported** from `src/`.
|
|
28
|
+
|
|
29
|
+
## Authority and scope
|
|
30
|
+
|
|
31
|
+
Augusto owns all main-path empirical/paper choices:
|
|
32
|
+
|
|
33
|
+
- construct definition, unit/universe, score matrix and scoring protocol, and measurement menu;
|
|
34
|
+
- empirical nomological network `R`, anchors, directions, thresholds, tolerance policy, and beta choice;
|
|
35
|
+
- paper claims, reporting placement, H5/public-baseline choice, release posture, tag, and PyPI publication.
|
|
36
|
+
|
|
37
|
+
Agents may author oracle networks only for synthetic DGPs. Do not invent an empirical network to make H5 or a paper table move. The existing spam audit is intermediate stress evidence, **not H5**.
|
|
38
|
+
|
|
39
|
+
The current paper protocol (`docs/16_Paper_Protocol_Freeze.md`) is **LOCKED AS PROVISIONAL SYNTHETIC-ONLY**. It does not authorize empirical/H5 work, engine changes, pushes, tags, PyPI publication, or a release claim by implication. A dated amendment (`docs/16` §8, 2026-08-04) opens the empirical box **for the designated H5 Trust evaluation only**; its first frozen run (n=35) is **preliminary paper-facing evidence** (`reports/summaries/h5_trust_evidence_summary.json`) — a checkpoint, not a release. Final paper lock, tag, and PyPI remain Augusto's.
|
|
40
|
+
|
|
41
|
+
## Current posture
|
|
42
|
+
|
|
43
|
+
- Package version: `2.0.0` (v2.0 release; PyPI publication in progress).
|
|
44
|
+
- v1.1 inference layer shipped and **tagged `v1.1.0`** (2026-08-04, `fce31c8`); PyPI release pending.
|
|
45
|
+
- **v2.0 measure discipline ENTRY COMPLETE** (`docs/18_Measure_Discipline_Plan.md`): δ-grid, evaluator growth (`mean_order`/`rank_agree`/`ols_coef`), θ-anchor artifacts all shipped (2026-08-05); DONE phase in progress (publication authorized, tutorial pending).
|
|
46
|
+
- `reports/summaries/v1_1_package_synth_summary.json` is package smoke evidence (seeds `0..4`).
|
|
47
|
+
- `reports/summaries/v1_1_protocol_synth_mc50_summary.json` is a distinct provisional synthetic-only MC50 proof artifact (seeds `0..49`). It is not H5 or a full empirical paper lock.
|
|
48
|
+
- `reports/runs/` contains reproducible bulk outputs and is ignored. Allow-listed proof summaries under `reports/summaries/` are tracked.
|
|
49
|
+
- Intermediate real-world audits live under `evals/realworld/` (`spam_validity` text; `calhousing_validity` tabular). Both are **intermediate / not H5**; their READMEs record capability boundaries (no LLM calls in the engine, no missingness imputation, descriptive correlations only). Do not cite them as paper evidence.
|
|
50
|
+
|
|
51
|
+
## Where truth lives
|
|
52
|
+
|
|
53
|
+
| Source | Use |
|
|
54
|
+
|---|---|
|
|
55
|
+
| `README.md` | Orientation, scope, current package posture |
|
|
56
|
+
| `docs/PROJECT_MANIFEST.md` | Machine-readable project state and locks |
|
|
57
|
+
| `docs/12_Decision_Engineering_Log.md` | Append-only engineering/scope decisions |
|
|
58
|
+
| `docs/13_Evaluations_Log.md` | Evidence interpretations and artifact pointers |
|
|
59
|
+
| `docs/15_MVP_Release_Checklist.md` | Release-review input; not release authority |
|
|
60
|
+
| `docs/16_Paper_Protocol_Freeze.md` | Paper-facing locks, open fields, and provenance rule |
|
|
61
|
+
| `docs/17_H5_Trust_Design.md` | H5 Trust design lock (run gated) |
|
|
62
|
+
| `docs/18_Measure_Discipline_Plan.md` | v2.0 scope box: δ-grid, evaluator growth, θ-anchor discipline |
|
|
63
|
+
| `pyproject.toml` | Package version, dependencies, lint/type/test configuration |
|
|
64
|
+
| `.github/workflows/ci.yml` | CI contract |
|
|
65
|
+
| `.gitignore` | Tracked proofs versus ignored run output |
|
|
66
|
+
|
|
67
|
+
If code and current policy documentation disagree, stop and surface the conflict; do not silently make code, docs, or theory conform.
|
|
68
|
+
|
|
69
|
+
## Working conventions
|
|
70
|
+
|
|
71
|
+
1. Read the relevant source, tests, and current decision/protocol docs before changing code.
|
|
72
|
+
2. Use strict TDD for behavioral source changes: write a focused test, observe RED, implement minimally, observe GREEN, then run the relevant module and full suite.
|
|
73
|
+
3. Keep implementation thin and observable. Do not add infrastructure, optional frameworks, or generic extension systems without a fixture and an explicit decision.
|
|
74
|
+
4. For docs/logs, preserve history: append a dated reversal or close-out rather than rewriting a prior decision.
|
|
75
|
+
5. Never commit ignored bulk run output. Verify proof-summary allow-lists before adding generated JSON.
|
|
76
|
+
6. Do not push, tag, publish, rewrite history, or modify `v0.1` without a task-specific explicit user decision.
|
|
77
|
+
|
|
78
|
+
## Commands
|
|
79
|
+
|
|
80
|
+
Bootstrap development dependencies:
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
uv sync --extra dev
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Run the local quality battery:
|
|
87
|
+
|
|
88
|
+
```bash
|
|
89
|
+
uv run ruff check src tests tools
|
|
90
|
+
uv run mypy src
|
|
91
|
+
uv run pytest -q --tb=short
|
|
92
|
+
uv run cvprofiles --version
|
|
93
|
+
git diff --check
|
|
94
|
+
git rev-parse 'v0.1^{}'
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
For CLI changes, preserve the contract: stdout is machine-readable JSON; human status messages belong on stderr. Empty `M*` is an exit-0 success path.
|
|
98
|
+
|
|
99
|
+
Before a commit, inspect `git status -sb`, the diff, and relevant on-disk artifacts. Report measured outputs, not remembered numbers.
|
|
100
|
+
|
|
101
|
+
## Explicit non-goals unless a dated decision reopens them
|
|
102
|
+
|
|
103
|
+
- foundation-model training;
|
|
104
|
+
- new annotation campaigns as the main route;
|
|
105
|
+
- SAE/hypothesis-generation core;
|
|
106
|
+
- full PPI/MARS reimplementation;
|
|
107
|
+
- agent-based-model/SCA product work as the spine;
|
|
108
|
+
- proprietary APIs for paper-reproducible work;
|
|
109
|
+
- GUI/SaaS/heavy infrastructure;
|
|
110
|
+
- “automate all empirical economics.”
|
|
111
|
+
|
|
112
|
+
When in doubt, favor a small, auditable artifact over a clever abstraction. The point is disciplined measurement, not an elaborate machine for avoiding decisions.
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable project milestones live here. Detailed decisions → `docs/12_Decision_Engineering_Log.md`. Eval learnings → `docs/13_Evaluations_Log.md`.
|
|
4
|
+
|
|
5
|
+
Format: keep newest first.
|
|
6
|
+
|
|
7
|
+
## [1.1.0] — 2026-08-04 (MVP tag `v1.1.0`; PyPI publication separate)
|
|
8
|
+
|
|
9
|
+
### Engine — v1.1 inference layer (shipped on `main` as dev `1.1.0a1`)
|
|
10
|
+
|
|
11
|
+
- Bootstrap over observational units (percentile band over non-empty replicates; empty/degenerate counts reported) — **M6 inference layer**
|
|
12
|
+
- Deterministic θ-grid sensitivity surface (diagnostic; λ scales threshold magnitudes only; headline stays λ=1.0; grid excluded from freeze preimage)
|
|
13
|
+
- Additive pipeline, CLI (`--n-boot`, `--theta-grid`), JSON/HTML audit panels, and stale-layer cleanup
|
|
14
|
+
- Package-native battery and inference evidence: `reports/summaries/v1_1_package_synth_summary.json`
|
|
15
|
+
- MVP release checklist: `docs/15_MVP_Release_Checklist.md` (feeds release-review chat)
|
|
16
|
+
- Verification close-out: local ruff clean, 121 tests passed, version smoke and tag checks passed; GitHub Actions green confirmed by Augusto; no tag or PyPI publication attempted
|
|
17
|
+
- PyPI name availability checked (HTTP 404); publication decision remains open and is **not** part of v1.1
|
|
18
|
+
- Independent MC50 proof audit: `tools/verify_v11_protocol_synth_mc50.py` + `tests/test_v11_protocol_synth_mc50_audit.py` validate the provisional synthetic-only protocol table in a read-only pass
|
|
19
|
+
- Strict typing enforcement: `uv run mypy src` is green and enforced in CI; `ruff` now also lints `tools/` in CI
|
|
20
|
+
- Protocol/release documentation reconciliation: README/manifest/open-questions now reflect the provisional synthetic-only lock (`docs/16`), the MC50 proof table, and the new `AGENTS.md`; release posture unchanged
|
|
21
|
+
|
|
22
|
+
### H5 Trust — first paper-facing evaluation (2026-08-04)
|
|
23
|
+
|
|
24
|
+
- **Design lock:** `docs/17_H5_Trust_Design.md` — country-level generalized trust; WVS7 × GPS × WDI/WGI; pinned network (gps_trust 0.3 / rule_of_law 0.3 / gini −0.1); δ=0; β=corr_y on log GDP pc; claims boundary
|
|
25
|
+
- **First frozen build:** n=35 countries; `evals/h5_trust/build_dataset.py` (masking, aggregation, coverage rule, canonical hashes); reproducible `--fetch-wdi`/`--fetch-wgi`
|
|
26
|
+
- **Preliminary paper-facing evidence (owner-approved 2026-08-04):** M\*={m_trust_general, m_trust_in_group}, [L,U]=[0.371, 0.624], FA=0, cold H4; θ-grid empties at λ≥1.5; bootstrap band [0.174, 0.752] with 17.5% empty replicates
|
|
27
|
+
- Tracked summary: `reports/summaries/h5_trust_evidence_summary.json`; auditors `tools/verify_h5_trust.py` + `tools/verify_v11_protocol_synth_mc50.py`
|
|
28
|
+
- **Boundary:** preliminary checkpoint; final paper lock, tag, PyPI, and push remain Augusto's
|
|
29
|
+
|
|
30
|
+
### Release posture (2026-08-04)
|
|
31
|
+
|
|
32
|
+
- **Packaging fix:** removed duplicate hatchling force-include that broke `uv build` (report.html.j2 added twice); wheel now builds cleanly
|
|
33
|
+
- **Wheel verified from a fresh venv:** template packaged, console script works, H5 run reproduces bit-identical M\* and [L,U]
|
|
34
|
+
- Release checklist recaptured: 157 tests passed, ruff/mypy clean, `v0.1` intact; tag `v1.1.0` and PyPI publication remain Augusto + release-review decisions
|
|
35
|
+
|
|
36
|
+
### Tag `v1.1.0` (2026-08-04)
|
|
37
|
+
|
|
38
|
+
- Annotated tag `v1.1.0` created and pushed (Augusto's explicit decision) — symbolizes the MVP: v1.0 spine + v1.1 inference layer + H5 Trust preliminary evidence + packaging/CI fixes
|
|
39
|
+
- Tag-as-symbolization convention (v0.1 precedent): wheel remains dev `1.1.0a1`; PyPI publication is a separate decision with version alignment at publish time
|
|
40
|
+
|
|
41
|
+
## [Unreleased]
|
|
42
|
+
|
|
43
|
+
### Planned
|
|
44
|
+
|
|
45
|
+
- Measure discipline: δ-grid, evaluator registry growth (`mean_order`/`rank_agree`/`ols_coef`), θ-anchor documentation discipline
|
|
46
|
+
- H5 robustness checks + extensions (v2.0 roadmap)
|
|
47
|
+
|
|
48
|
+
## [1.0.0a1] — 2026-08-01 (spine shipped on `main`)
|
|
49
|
+
|
|
50
|
+
**v1.0 thin first-principles spine** — SCORE → RESTRICT → IDENTIFY → thin REPORT with `[L,U]=min/max B*`, freeze hashes, mini fixture, package-native synth battery, minimal CI, and an intermediate real-world spam audit (not H5). Merged to `main` @ `3be6367`. Tag decision owned by release-review chat.
|
|
51
|
+
|
|
52
|
+
### Added
|
|
53
|
+
- **M9 / minimal CI:** `.github/workflows/ci.yml` — uv sync, ruff, pytest, CLI smoke, import-graph hygiene, mini fixture SCORE→REPORT on Python 3.11/3.12
|
|
54
|
+
- **Intermediate real-world audit:** `evals/realworld/spam_validity/` — 20newsgroups multi-measure spamminess stress; `verify_audit.py` exit 0; FA=0; harsh empty; cold H4; **not** main-path H5
|
|
55
|
+
- **M8 / G8:** package-native synth harness (`src/cvprofiles/synth/`); battery drives real SCORE→RESTRICT→IDENTIFY; H1a/H1b/H3/H4 **green** on seeds `0..4`, \(n=1000\); H1_latent diagnostic only (attenuation → 0); proof `reports/summaries/v1_0_package_synth_summary.json`; museum unimported
|
|
56
|
+
- **M7 / G7:** thin REPORT (HTML/JSON); `run_profile` SCORE→REPORT composition; CLI `cvprofiles run` (stdout JSON / stderr crumbs); empty-\(M^*\) first-class; e2e on mini_v1 oracle + harsh
|
|
57
|
+
- **M5 / G5:** \(\beta=\mathrm{corr}_y\); \([L,U]=\min/\max B^*\) on survivors only; empty/point-ID flags; `range.json` notes bootstrap deferred to v1.1
|
|
58
|
+
- **M4 / G4:** IDENTIFY slacks (`corr_min`, `corr_sign`); \(M^*\) + rejection reasons; harsh empty fixture (`network_harsh.yaml`, \(\theta=0.999\)); FA=0 on `m_slop`
|
|
59
|
+
- **M3 / G3:** RESTRICT load/bind network+beta; `RestrictError` at IO boundary; golden network/beta hashes
|
|
60
|
+
- **M2 / G2:** SCORE pipeline (`run_score` / `write_score_artifacts`); default normalization `none`; optional `zscore_measures` on measures only; fail-loud on missing cols / empty / dup unit_id / non-finite; golden `scores_hash` match on mini_v1
|
|
61
|
+
- **M1 / G1:** installable `src/cvprofiles` (`1.0.0a1`) + `pyproject.toml` + thin CLI
|
|
62
|
+
- Pydantic v2 schemas (scores / network / beta / freeze run manifests)
|
|
63
|
+
- Freeze hash + `run_id` util (`cvprofiles.freeze`); algorithm locked in `docs/12`
|
|
64
|
+
- Mini fixture `data/fixtures/mini_v1/` + golden `expected_freeze.json`
|
|
65
|
+
- Contract tests (schemas, freeze bit-stability, import-graph hygiene, mini load)
|
|
66
|
+
- **Phase 0 (v1.0 sprint start):** scope box — thin first-principles spine; M6 bootstrap/θ-grid deferred to v1.1; no M10 this sprint
|
|
67
|
+
- Past-tense docs: public repo + tag `v0.1` @ `fb62b48` live; G7/G8 re-enter from G5 for v1.0; H1a/H1b gates + H1_latent diagnostic pointers
|
|
68
|
+
- `docs/14_Researcher_Input_Guide.md` — DRAFT process guide for composites, anchors, SCORE/RESTRICT prep (not part of tag `v0.1`)
|
|
69
|
+
|
|
70
|
+
### Planned (later)
|
|
71
|
+
- Paper prereg freeze
|
|
72
|
+
- Public baseline H5 (USER-authored network only)
|
|
73
|
+
- δ-grid, `mean_order`/`rank_agree` evaluators, `ols_coef`, LaTeX report (post-MVP backlog)
|
|
74
|
+
|
|
75
|
+
## [0.1.0] — 2026-08-01
|
|
76
|
+
|
|
77
|
+
**Symbolizes v0.1:** methods knowledge base + green synthetic PoC (museum monolith).
|
|
78
|
+
**Not** a PyPI release. **Not** the package API.
|
|
79
|
+
|
|
80
|
+
### Added
|
|
81
|
+
- Project scaffold under `~/Hermes/Projects/cvprofiles`
|
|
82
|
+
- Methods-first documentation suite (`docs/01`–`13`, `PROJECT_MANIFEST.md`)
|
|
83
|
+
- Dual live logs (decision/engineering + evaluations)
|
|
84
|
+
- MIT `LICENSE`
|
|
85
|
+
- **`evals/synthetic/v0_poc.py`** monolith (`POC_VERSION=v0_1_poc`) — SCORE→REPORT synthetic battery with gate check
|
|
86
|
+
- Proof summary `reports/summaries/v0_1_poc_summary.json`
|
|
87
|
+
|
|
88
|
+
### Fixed / learned
|
|
89
|
+
- **v0 → v0.1 hygiene:** distinct `oracle_with_slop`; true empty `all_invalid`; cold determinism; near_miss fails by design
|
|
90
|
+
- **H1 split locked:** H1a/H1b gates; H1_latent diagnostic only (attenuation); no \(\theta\)-loosening
|
|
91
|
+
- v0.1 battery **exit 0** (FA=0, empty honesty, anchor retention, cold OK)
|
|
92
|
+
|
|
93
|
+
### Explicit non-contents
|
|
94
|
+
- No `src/cvprofiles` package layout
|
|
95
|
+
- No bootstrap / θ-grid inference layer
|
|
96
|
+
- No real-data baseline / H5 network
|
|
97
|
+
- No LLM inside the engine
|
cvprofiles-2.0.0/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Augusto Gonzalez Bonorino
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: cvprofiles
|
|
3
|
+
Version: 2.0.0
|
|
4
|
+
Summary: Construct-validity profiles: admissible measurement sets and construct-identified ranges
|
|
5
|
+
Project-URL: Homepage, https://github.com/Bonorinoa/cvprofiles
|
|
6
|
+
Project-URL: Repository, https://github.com/Bonorinoa/cvprofiles
|
|
7
|
+
Author: Augusto Gonzalez Bonorino
|
|
8
|
+
License: MIT
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Keywords: construct-validity,econometrics,measurement,partial-identification
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Intended Audience :: Science/Research
|
|
13
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
17
|
+
Classifier: Topic :: Scientific/Engineering
|
|
18
|
+
Requires-Python: >=3.11
|
|
19
|
+
Requires-Dist: jinja2>=3.1
|
|
20
|
+
Requires-Dist: numpy>=1.26
|
|
21
|
+
Requires-Dist: pandas>=2.0
|
|
22
|
+
Requires-Dist: pyarrow>=14.0
|
|
23
|
+
Requires-Dist: pydantic>=2.0
|
|
24
|
+
Requires-Dist: pyyaml>=6.0
|
|
25
|
+
Requires-Dist: scipy>=1.11
|
|
26
|
+
Requires-Dist: typer>=0.12
|
|
27
|
+
Provides-Extra: dev
|
|
28
|
+
Requires-Dist: mypy>=1.8; extra == 'dev'
|
|
29
|
+
Requires-Dist: pandas-stubs>=2.0; extra == 'dev'
|
|
30
|
+
Requires-Dist: pytest>=8.0; extra == 'dev'
|
|
31
|
+
Requires-Dist: ruff>=0.4; extra == 'dev'
|
|
32
|
+
Requires-Dist: types-pyyaml>=6.0; extra == 'dev'
|
|
33
|
+
Description-Content-Type: text/markdown
|
|
34
|
+
|
|
35
|
+
# cvprofiles
|
|
36
|
+
|
|
37
|
+
**Construct-validity profiles for cheap multi-measure AI operationalizations.**
|
|
38
|
+
|
|
39
|
+
Open, high-observability research tooling that treats construct validity as **partial identification over a menu of measurement functions**, disciplined by a researcher-authored nomological network. The engine returns an admissible measurement set \(M^*\) and a construct-identified range \([L,U]\) for a target functional \(\beta(\cdot)\). Empty sets and wide ranges are scientific features, not product failures.
|
|
40
|
+
|
|
41
|
+
| | |
|
|
42
|
+
|---|---|
|
|
43
|
+
| **Version** | **v0.1** tagged (methods KB + museum PoC); **v1.1.0 tagged** (MVP: spine + inference layer + H5 Trust evidence); **v2.0.0 release (publication in progress)** |
|
|
44
|
+
| **Status** | Public repo live; tags `v0.1` @ `fb62b48` and **`v1.1.0`** (MVP, 2026-08-04) frozen; protocol draft locked **provisional synthetic-only**; H5 Trust design locked (`docs/17`), first frozen run accepted as **preliminary paper-facing evidence** (n=35, [L,U]=[0.371,0.624]); PyPI publication pending |
|
|
45
|
+
| **Type** | Academic methods tool (Python package + paper) |
|
|
46
|
+
| **Owner** | Augusto Gonzalez Bonorino |
|
|
47
|
+
| **License** | MIT |
|
|
48
|
+
| **GitHub** | https://github.com/Bonorinoa/cvprofiles |
|
|
49
|
+
| **CI** | [](https://github.com/Bonorinoa/cvprofiles/actions/workflows/ci.yml) |
|
|
50
|
+
| **Hermes profile** | `cvprofiles` |
|
|
51
|
+
| **Path** | `~/Hermes/Projects/cvprofiles` |
|
|
52
|
+
| **PoC** | `evals/synthetic/v0_poc.py` (`v0_1_poc`) — **museum monolith**, not package |
|
|
53
|
+
| **Proof summary** | `reports/summaries/v0_1_poc_summary.json`, `v1_0_package_synth_summary.json`, `v1_1_package_synth_summary.json`, `v1_1_protocol_synth_mc50_summary.json` (provisional synthetic-only protocol table; not H5 / not a paper result) |
|
|
54
|
+
|
|
55
|
+
## Thesis spine (one paragraph)
|
|
56
|
+
|
|
57
|
+
Researcher supplies unit×measure scores (SCORE). Researcher authors a nomological network \(R\) with thresholds \(\theta\) and a target \(\beta(\cdot)\) (RESTRICT). Engine computes sample slacks, keeps admissible measures \(M^*\), maps survivors through \(\beta\), and reports \(B^* / [L,U]\) (IDENTIFY). **v1.0:** \([L,U]=\min/\max B^*\) only. Bootstrap and \(\theta\)-sensitivity are **v1.1**. Measure discipline (δ-grid, evaluator growth, θ-anchor artifacts) is **v2.0**. Audit trail in HTML/JSON (LaTeX later) a non-coder can steer (REPORT). Engine is **score-agnostic and model-free**. No LLM lives inside the engine.
|
|
58
|
+
|
|
59
|
+
## Document map
|
|
60
|
+
|
|
61
|
+
Start here, then read in order:
|
|
62
|
+
|
|
63
|
+
| Doc | Purpose |
|
|
64
|
+
|---|---|
|
|
65
|
+
| [`docs/01_Project_Overview.md`](docs/01_Project_Overview.md) | Goals, users, scope, hard non-goals |
|
|
66
|
+
| [`docs/02_System_Architecture.md`](docs/02_System_Architecture.md) | 4-state machine, IO contracts, determinism |
|
|
67
|
+
| [`docs/03_Methodology.md`](docs/03_Methodology.md) | Menu, slacks, \(M^*\), \(B^*\), inference stance |
|
|
68
|
+
| [`docs/04_Synthetic_DGPs.md`](docs/04_Synthetic_DGPs.md) | Calibrated DGPs + four debug metrics |
|
|
69
|
+
| [`docs/05_Pre_Registration.md`](docs/05_Pre_Registration.md) | Draft H1–H5 (user owns theory) |
|
|
70
|
+
| [`docs/06_Tech_Stack.md`](docs/06_Tech_Stack.md) | Language, numeric stack, packaging |
|
|
71
|
+
| [`docs/07_Software_Development_Strategy.md`](docs/07_Software_Development_Strategy.md) | TDD, gates, agent workflow |
|
|
72
|
+
| [`docs/08_Observability_and_Evaluations.md`](docs/08_Observability_and_Evaluations.md) | Artifacts, metrics, report contract |
|
|
73
|
+
| [`docs/09_MVP_Plan.md`](docs/09_MVP_Plan.md) | Milestones and build order |
|
|
74
|
+
| [`docs/10_Open_Questions.md`](docs/10_Open_Questions.md) | Explicit deferrals |
|
|
75
|
+
| [`docs/11_Glossary.md`](docs/11_Glossary.md) | Notation |
|
|
76
|
+
| [`docs/12_Decision_Engineering_Log.md`](docs/12_Decision_Engineering_Log.md) | **LIVE** engineering decisions |
|
|
77
|
+
| [`docs/13_Evaluations_Log.md`](docs/13_Evaluations_Log.md) | **LIVE** eval runs and learnings |
|
|
78
|
+
| [`docs/14_Researcher_Input_Guide.md`](docs/14_Researcher_Input_Guide.md) | Composites, anchors, SCORE/RESTRICT prep |
|
|
79
|
+
| [`docs/15_MVP_Release_Checklist.md`](docs/15_MVP_Release_Checklist.md) | MVP release checklist (feeds release-review chat) |
|
|
80
|
+
| [`docs/16_Paper_Protocol_Freeze.md`](docs/16_Paper_Protocol_Freeze.md) | Paper-facing locks, open fields, provenance rule |
|
|
81
|
+
| [`docs/17_H5_Trust_Design.md`](docs/17_H5_Trust_Design.md) | H5 Trust design (LOCKED as design; run gated) |
|
|
82
|
+
| [`docs/18_Measure_Discipline_Plan.md`](docs/18_Measure_Discipline_Plan.md) | v2.0 scope box: δ-grid, evaluator growth, θ-anchor discipline |
|
|
83
|
+
| [`tutorials/cvprofiles_tutorial.ipynb`](tutorials/cvprofiles_tutorial.ipynb) | Independent tutorial: synthetic walk-through + H5 replication |
|
|
84
|
+
| [`docs/PROJECT_MANIFEST.md`](docs/PROJECT_MANIFEST.md) | Machine-readable index |
|
|
85
|
+
|
|
86
|
+
## Roadmap
|
|
87
|
+
|
|
88
|
+
### v1.0 (this sprint — thin first-principles spine)
|
|
89
|
+
|
|
90
|
+
1. **Schemas + freeze contract** — typed score matrix, network, \(\beta\), run manifest
|
|
91
|
+
2. **SCORE → RESTRICT → IDENTIFY** — slacks, \(M^*\), \([L,U]=\min/\max B^*\) (no bootstrap)
|
|
92
|
+
3. **Thin REPORT** — HTML/JSON; empty \(M^*\) is a clean success
|
|
93
|
+
4. **Synth harness re-impl** under package/tests (H1a / H2 / H3 / H4; H1_latent diagnostic)
|
|
94
|
+
5. **Installable package + minimal CI**
|
|
95
|
+
|
|
96
|
+
### v1.1 (tagged `v1.1.0` 2026-08-04 — MVP; PyPI publication pending)
|
|
97
|
+
|
|
98
|
+
- Units-only bootstrap with conservative, additive percentile diagnostics
|
|
99
|
+
- Deterministic θ-grid sensitivity surface; headline remains [L,U]=min/max B*
|
|
100
|
+
- Pipeline, CLI, JSON/HTML audit panels, package-native evidence, and minimal CI
|
|
101
|
+
- Version `2.0.0`; tag `v1.1.0` live; v2.0 measure discipline ENTRY complete; PyPI publication in progress
|
|
102
|
+
- Provisional synthetic-only protocol lock (`docs/16`) with an audited MC50 proof table (seeds `0..49`); H5 Trust design locked (`docs/17`), first frozen run (n=35) accepted as **preliminary paper-facing evidence**; empirical/paper inputs remain Augusto-owned
|
|
103
|
+
|
|
104
|
+
### Remaining backlog
|
|
105
|
+
|
|
106
|
+
- **Paper-facing protocol freeze:** [`docs/16_Paper_Protocol_Freeze.md`](docs/16_Paper_Protocol_Freeze.md) — construct, score matrix, menu, researcher-authored \(R\), \(\theta\), \(\delta\), \(\beta\), and evidence posture (synthetic-only portion currently locked provisional)
|
|
107
|
+
- **M10 / H5:** country-level generalized trust baseline — design **LOCKED** (`docs/17`); first frozen run (n=35) accepted as **preliminary paper-facing evidence** (`reports/summaries/h5_trust_evidence_summary.json`); final paper lock + release remain Augusto's
|
|
108
|
+
- **v2.0 measure discipline (in progress):** see [`docs/18_Measure_Discipline_Plan.md`](docs/18_Measure_Discipline_Plan.md) — δ-grid tolerance layer, `mean_order`/`rank_agree`/`ols_coef` evaluators, θ-anchor artifacts, and an H5-replication tutorial. LaTeX report remains later backlog.
|
|
109
|
+
|
|
110
|
+
See [`docs/09_MVP_Plan.md`](docs/09_MVP_Plan.md) for the locked v1.0 scope box and [`docs/15_MVP_Release_Checklist.md`](docs/15_MVP_Release_Checklist.md) for the v1.1 handoff checklist.
|
|
111
|
+
|
|
112
|
+
## Hard non-goals (unless reopened in the decision log)
|
|
113
|
+
|
|
114
|
+
- Foundation-model training
|
|
115
|
+
- New human annotation campaigns as the main path
|
|
116
|
+
- Hypothesis-generation / SAE pipelines as thesis core
|
|
117
|
+
- Full PPI/MARS reimplementation as co-equal deliverable
|
|
118
|
+
- LLM-agent ABMs / SCA product work as this spine
|
|
119
|
+
- Proprietary API dependence for paper-reproducible results
|
|
120
|
+
- “Automate all of empirical economics” platforms
|
|
121
|
+
|
|
122
|
+
## Working convention
|
|
123
|
+
|
|
124
|
+
Progress is read from **on-disk artifacts** (slacks, \(M^*\), ranges, `report.html`) and the two live logs — not from git archaeology alone. Paper numbers come only from **frozen score matrices + pinned network + fixed seed**.
|
|
125
|
+
|
|
126
|
+
## Repo status
|
|
127
|
+
|
|
128
|
+
- **Tag `v0.1` @ `fb62b48` (live, immovable):** documentation suite + museum synthetic PoC with green gates.
|
|
129
|
+
Release: https://github.com/Bonorinoa/cvprofiles/releases/tag/v0.1
|
|
130
|
+
- **`main`:** v1.0 spine through M9 (shipped as `1.0.0a1`) + intermediate spam audit + v1.1 inference layer, merged.
|
|
131
|
+
- **v1.1.0 tagged (2026-08-04):** MVP — v1.0 spine + v1.1 inference layer (units bootstrap + θ-grid) + H5 Trust preliminary evidence. Release version `2.0.0`; PyPI publication in progress.
|
|
132
|
+
- **Not yet:** PyPI release. **v2.0 measure discipline in progress** (`docs/18`): δ-grid, evaluator growth, θ-anchor artifacts, H5-replication tutorial.
|
|
133
|
+
|
|
134
|
+
### Install (dev)
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
uv sync --extra dev
|
|
138
|
+
uv run pytest -q
|
|
139
|
+
uv run cvprofiles --version
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
### Museum PoC (expects exit 0; do not import into `src/`)
|
|
143
|
+
|
|
144
|
+
```bash
|
|
145
|
+
uv venv --python 3.11 .venv
|
|
146
|
+
uv pip install --python .venv/bin/python numpy pandas
|
|
147
|
+
.venv/bin/python evals/synthetic/v0_poc.py
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
- Progress is read from on-disk artifacts + `docs/12` / `docs/13`, not git archaeology alone.
|