pen-stack 6.5.0__tar.gz → 6.7.0__tar.gz

{pen_stack-6.5.0 → pen_stack-6.7.0}/CHANGELOG.md

@@ -3,6 +3,71 @@
 All notable changes to PEN-STACK are documented here. This file follows
 [Keep a Changelog](https://keepachangelog.com/) and the program's phase structure.
 
+## [6.7.0] - 2026-06-19 - PEN-EXPRESS: learned, trained-conformal Stage H + TPE-Bench
+
+**MINOR feature release.** Upgrades the digital twin's Stage H expression/outcome layer from a validation-failing
+closed-form **heuristic** to a **learned, trained-conformal, decomposable** position-effect model — and ships the
+held-out benchmark the expression capability never had. Wrap, don't rebuild: extends `twin` + `wgenome.uncertainty`
++ `wgenome.ood` + `benchmarks`. No fabrication: every metric is from a real CV run on real TRIP supervision, and
+the cross-cell-type transfer claim is **data-gated**, never faked.
+
+### Added — the learned model + trained conformal (WS-EXPRESS2: WS-D/M/U)
+- `pen_stack/twin/data/position_effect.py` — unified position-effect schema + dataset registry with **verified
+  accessions/DOIs** (TRIP live; PatchMPRA/MPIRE/lentiMPRA/Leemans registered + honestly `available=False` until
+  fetched), z-normalization within (dataset × cassette), domain-blocked + held-out-cell-type splits + leakage check.
+- `pen_stack/twin/position_effect.py` — `PositionEffectModel` (factored `f_cassette` + `g_context`, LightGBM),
+  `evaluate()` (chromosome-blocked CV vs the v3.x durability head + cassette-only, paired-bootstrap CIs,
+  separability), split-conformal calibration (`ConformalRegressor`, chromosome-Mondrian, OOD-widened),
+  `predict_stage_h()` serving seam. **Result (real TRIP):** expression ρ **0.428 → 0.469** (CI excludes 0);
+  held-out conformal coverage **0.885** vs 0.90 nominal.
+- `configs/twin/position_effect_conformal.json` — the shipped calibration (qhat + N + held-out coverage).
+- `scripts/p1_build_position_effect.py` — regenerates the model + conformal artifacts (real CV report).
+
+### Changed — Stage H integration (WS-I)
+- `pen_stack/twin/outcome.py` — when a chromatin context is supplied and the artifact is present, `predict_outcome`
+  serves the **learned trained-conformal** interval + `p_silenced` + OOD tier (`position_effect` block,
+  `stage_h_mode`); with no context/artifact it falls back to the heuristic band — **backward compatible** (the
+  v5.9 relative-scale contract is intact).
+
+### Added — TPE-Bench + controls (WS-B / WS-V)
+- `benchmarks/position_effect/` — TPE-Bench: a **sealed, SHA-locked** held-out-chromosome track + baseline
+  leaderboard (cassette-only / durability head / PEN-EXPRESS factored), submission harness. Leave-one-cell-type-out
+  transfer track scaffolded + **data-gated** (no fabricated transfer number).
+- `pen_stack/validate/{expr_controls,known_biology_expr,heldout_celltype_expr}.py` — label-shuffle→chance control,
+  H3K9me3-heterochromatin→silencing recovery, the data-gated transfer harness.
+- `tests/unit/test_ws_pe.py` — CI-safe (synthetic planted signal); the real-TRIP claim runs on a checkout, skips in CI.
+
+### Honesty
+- Public data **cannot** flip expression to ✅ (the v6.5 wall) — v6.7 ships the learned+calibrated upgrade + the
+  benchmark + the honest data-gating, not a manufactured checkmark. Cross-cell-type transfer needs the additional
+  human datasets (a data-acquisition step), reported as such. Wet-lab validation omitted by scope.
+
+## [6.6.0] - 2026-06-16 - License-clean provenance (COSMIC → CancerMine)
+
+**MINOR — provenance refactor, no new science, no capability lost.** The shipped artifact now sources the
+oncogene/TSG/driver list from **CancerMine (CC0)** instead of COSMIC Cancer Gene Census (free for academia but
+**no-redistribution**). Copyright protects the *compiled database*, not the *fact* that a gene is an oncogene — so
+sourcing the *list* from a CC0 compilation removes all licensing doubt while keeping the same capability. Prep for
+the BioFirewall release, whose open repo vendors PEN-STACK's hazard data. Workstream WS-LIC + WS-CM + WS-REGEN.
+
+### Changed
+- `pen_stack/data/ingest_safety_annot.py` — `load_cancermine()` (CC0) is the **default** oncogene/TSG source
+  (HUGO→coords via GENCODE, `--min-citations` precision knob); `load_cosmic()` stays available but **off by
+  default** (bring-your-own-license, local enrichment only).
+- `configs/genotoxicity_oracle.yaml` — regenerated from CancerMine; provenance + DOIs updated (CancerMine
+  10.1038/s41592-019-0422-y). `safety_{ct}.pkl` + the Writable-Genome atlas regenerated on CancerMine features
+  (re-deposited on Zenodo, superseding the COSMIC-derived deposit).
+
+### Added
+- **`DATA_LICENSES.md`** — every source × license × redistribution-status × where-used.
+- `tests/unit/test_data_licenses.py` — **CI license gate**: fails if a restricted source (COSMIC/OncoKB) is the
+  shipped derived-data source or a raw restricted gene-list is committed; CancerMine is the default.
+- `scripts/fetch_licensed_sources.py` — bring-your-own-license fetcher for COSMIC/OncoKB (local-only, validation).
+
+### Honesty
+- Metrics may shift (CancerMine has broader coverage than CGC) — reported, not hidden. The genotoxicity axis is a
+  **mechanism-grounded proxy (🟡, not outcome-validated)** before and after; the swap changes only the *source*.
+
 ## [6.5.0] - 2026-06-15 - Comprehensive expression model + honest proxy-validation pass
 
 **MINOR feature release.** Two threads, one principle (no fabrication):

{pen_stack-6.5.0 → pen_stack-6.7.0}/CITATION.cff

@@ -1,8 +1,8 @@
 cff-version: 1.2.0
 message: "If you use PEN-STACK, please cite it as below."
 title: "PEN-STACK: open infrastructure for genome writing"
-version: 6.5.0
-date-released: 2026-06-12
+version: 6.7.0
+date-released: 2026-06-19
 authors:
   - family-names: "Mahaboob Ali"
     given-names: "Anees Ahmed"

{pen_stack-6.5.0 → pen_stack-6.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pen-stack
-Version: 6.5.0
+Version: 6.7.0
 Summary: Open infrastructure for genome writing: the Writable Genome atlas, the Writer Atlas, and the Write Planner.
 Author-email: Anees Ahmed Mahaboob Ali <ahmedaneesm@gmail.com>
 License: MIT
@@ -90,7 +90,7 @@ every design against rule-grounded mechanism, reports calibrated confidence, cit
 [![codecov](https://codecov.io/gh/ahmedanees-m/pen-stack/branch/main/graph/badge.svg)](https://codecov.io/gh/ahmedanees-m/pen-stack)
 [![License: MIT](https://img.shields.io/badge/License-MIT-informational.svg)](LICENSE)
 [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/)
-[![Version](https://img.shields.io/badge/version-6.5.0-blue.svg)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-6.7.0-blue.svg)](CHANGELOG.md)
 [![Status](https://img.shields.io/badge/status-1.0%20First%20Stable-success.svg)](docs/STABILITY.md)
 [![Tests](https://img.shields.io/badge/tests-378%20passing-success.svg)](tests/)
 [![Lint: ruff](https://img.shields.io/badge/lint-ruff-purple.svg)](https://github.com/astral-sh/ruff)
@@ -148,6 +148,32 @@ demonstrated (v5.12) and the benchmark went public (v5.13).
 > unknown funnel remains — made legible (scope flags, known-unknowns, honest baselines, no fabrication), not
 > hidden.
 
+## What is new in v6.7 — PEN-EXPRESS (a learned, trained-conformal Stage H + TPE-Bench)
+
+Stage H's expression/outcome layer was the stack's weakest link: a closed-form **heuristic** that *failed*
+independent validation (ρ=0.12 vs Damdindorj 2014), with an interval the code itself labelled "**NOT a trained
+conformal interval**." v6.7 replaces it — **wrapping, not rebuilding** the digital twin:
+
+- **A learned cassette × context position-effect model** (`pen_stack/twin/position_effect.py`) — factored and
+  decomposable (`f_cassette` + `g_context`), trained on the **real TRIP supervision** (Akhtar 2013; GEO
+  GSE49806/49807). On chromosome-blocked CV it **beats the v3.x durability head** on expression (ρ **0.428 →
+  0.469**, paired-bootstrap CI excludes 0) and matches it on silencing — reported honestly, with the separability
+  result (additive `f_cassette + g_context` suffices at this N).
+- **Stage H is now trained-conformal.** The model ships a split-conformal calibration (`ConformalRegressor`,
+  chromosome-Mondrian, OOD-widened): **held-out coverage 0.885 vs 0.90 nominal** — the named gap, closed. When a
+  chromatin context is supplied, `predict_outcome` serves the calibrated interval + `p_silenced` + OOD tier; with
+  no context (or no artifact) it falls back to the heuristic — **backward compatible**.
+- **TPE-Bench** (`benchmarks/position_effect/`) — the held-out benchmark the expression capability never had: a
+  **sealed, SHA-locked** held-out-chromosome split + a baseline leaderboard (cassette-only / durability head /
+  PEN-EXPRESS). A leave-one-**cell-type**-out transfer track is scaffolded and **honestly data-gated** until the
+  additional human datasets (PatchMPRA / MPIRE / lentiMPRA / Leemans) are fetched — **no transfer number is
+  fabricated**. Controls: label-shuffle → chance; known-biology recovery (H3K9me3 ↑ → silencing ↑).
+
+The honest limit (the v6.5 wall, restated): public data alone cannot flip expression to ✅ — that needs measured
+outcome data at a power the literature does not provide. v6.7 ships the learned+calibrated upgrade + the
+benchmark; the green checkmark stays earned, not manufactured. See [docs/position_effect.md](docs/position_effect.md)
+and [docs/tpe_bench.md](docs/tpe_bench.md).
+
 ## What is new in v6.4 — Live Oracles (the foundation models actually execute)
 
 The foundation-model oracles now **run for real**, not just defer: **ViennaRNA** (in-process), **AlphaGenome**
@@ -976,18 +1002,6 @@ independently verified.
 - **Grounded services** - every quantitative answer comes from a validated tool call (never a language
   model); the living database never auto-edits the atlas; clinical directives are refused.
 
-## Papers and phases
-
-| # | Title | Phase | Status |
-|---|---|---|---|
-| 1 (flagship) | The Writable Genome: a predictive, writer-aware atlas of safe & durable insertion sites | 1 | complete |
-| 2 (platform) | PEN-STACK: unified open infrastructure for non-destructive genome writing | 2 | complete |
-| 3 (capstone) | The Write Planner: end-to-end inverse design of genomic writes | 3 | complete |
-| 4 (beachhead) | Genome-wide off-target prediction for RNA-guided bridge recombinases | 1.5 | complete |
-| M1 (v3.1) | Writable Genome hardened: strong baselines, AlphaGenome sequence + 3D structural-risk axis | v3.1 B,C,D,F | complete |
-| M2 (v3.1) | The Genome-Writing Bench + PEN-Agent: the writing-side benchmark and a grounded agent | v3.1 E | complete |
-| M3 (v3.1) | Multiplex translocation-risk + bridge-RNA guide QC | v3.1 G | complete |
-
 The v3.1 cycle (workstreams A-H) is recorded in `CHANGELOG.md`, `docs/positioning.md`, and the SHA-locked
 `prereg/ws_*.yaml`; preprint drafts are in `manuscripts/`.
 

{pen_stack-6.5.0 → pen_stack-6.7.0}/README.md

@@ -15,7 +15,7 @@ every design against rule-grounded mechanism, reports calibrated confidence, cit
 [![codecov](https://codecov.io/gh/ahmedanees-m/pen-stack/branch/main/graph/badge.svg)](https://codecov.io/gh/ahmedanees-m/pen-stack)
 [![License: MIT](https://img.shields.io/badge/License-MIT-informational.svg)](LICENSE)
 [![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue.svg)](https://www.python.org/)
-[![Version](https://img.shields.io/badge/version-6.5.0-blue.svg)](CHANGELOG.md)
+[![Version](https://img.shields.io/badge/version-6.7.0-blue.svg)](CHANGELOG.md)
 [![Status](https://img.shields.io/badge/status-1.0%20First%20Stable-success.svg)](docs/STABILITY.md)
 [![Tests](https://img.shields.io/badge/tests-378%20passing-success.svg)](tests/)
 [![Lint: ruff](https://img.shields.io/badge/lint-ruff-purple.svg)](https://github.com/astral-sh/ruff)
@@ -73,6 +73,32 @@ demonstrated (v5.12) and the benchmark went public (v5.13).
 > unknown funnel remains — made legible (scope flags, known-unknowns, honest baselines, no fabrication), not
 > hidden.
 
+## What is new in v6.7 — PEN-EXPRESS (a learned, trained-conformal Stage H + TPE-Bench)
+
+Stage H's expression/outcome layer was the stack's weakest link: a closed-form **heuristic** that *failed*
+independent validation (ρ=0.12 vs Damdindorj 2014), with an interval the code itself labelled "**NOT a trained
+conformal interval**." v6.7 replaces it — **wrapping, not rebuilding** the digital twin:
+
+- **A learned cassette × context position-effect model** (`pen_stack/twin/position_effect.py`) — factored and
+  decomposable (`f_cassette` + `g_context`), trained on the **real TRIP supervision** (Akhtar 2013; GEO
+  GSE49806/49807). On chromosome-blocked CV it **beats the v3.x durability head** on expression (ρ **0.428 →
+  0.469**, paired-bootstrap CI excludes 0) and matches it on silencing — reported honestly, with the separability
+  result (additive `f_cassette + g_context` suffices at this N).
+- **Stage H is now trained-conformal.** The model ships a split-conformal calibration (`ConformalRegressor`,
+  chromosome-Mondrian, OOD-widened): **held-out coverage 0.885 vs 0.90 nominal** — the named gap, closed. When a
+  chromatin context is supplied, `predict_outcome` serves the calibrated interval + `p_silenced` + OOD tier; with
+  no context (or no artifact) it falls back to the heuristic — **backward compatible**.
+- **TPE-Bench** (`benchmarks/position_effect/`) — the held-out benchmark the expression capability never had: a
+  **sealed, SHA-locked** held-out-chromosome split + a baseline leaderboard (cassette-only / durability head /
+  PEN-EXPRESS). A leave-one-**cell-type**-out transfer track is scaffolded and **honestly data-gated** until the
+  additional human datasets (PatchMPRA / MPIRE / lentiMPRA / Leemans) are fetched — **no transfer number is
+  fabricated**. Controls: label-shuffle → chance; known-biology recovery (H3K9me3 ↑ → silencing ↑).
+
+The honest limit (the v6.5 wall, restated): public data alone cannot flip expression to ✅ — that needs measured
+outcome data at a power the literature does not provide. v6.7 ships the learned+calibrated upgrade + the
+benchmark; the green checkmark stays earned, not manufactured. See [docs/position_effect.md](docs/position_effect.md)
+and [docs/tpe_bench.md](docs/tpe_bench.md).
+
 ## What is new in v6.4 — Live Oracles (the foundation models actually execute)
 
 The foundation-model oracles now **run for real**, not just defer: **ViennaRNA** (in-process), **AlphaGenome**
@@ -901,18 +927,6 @@ independently verified.
 - **Grounded services** - every quantitative answer comes from a validated tool call (never a language
   model); the living database never auto-edits the atlas; clinical directives are refused.
 
-## Papers and phases
-
-| # | Title | Phase | Status |
-|---|---|---|---|
-| 1 (flagship) | The Writable Genome: a predictive, writer-aware atlas of safe & durable insertion sites | 1 | complete |
-| 2 (platform) | PEN-STACK: unified open infrastructure for non-destructive genome writing | 2 | complete |
-| 3 (capstone) | The Write Planner: end-to-end inverse design of genomic writes | 3 | complete |
-| 4 (beachhead) | Genome-wide off-target prediction for RNA-guided bridge recombinases | 1.5 | complete |
-| M1 (v3.1) | Writable Genome hardened: strong baselines, AlphaGenome sequence + 3D structural-risk axis | v3.1 B,C,D,F | complete |
-| M2 (v3.1) | The Genome-Writing Bench + PEN-Agent: the writing-side benchmark and a grounded agent | v3.1 E | complete |
-| M3 (v3.1) | Multiplex translocation-risk + bridge-RNA guide QC | v3.1 G | complete |
-
 The v3.1 cycle (workstreams A-H) is recorded in `CHANGELOG.md`, `docs/positioning.md`, and the SHA-locked
 `prereg/ws_*.yaml`; preprint drafts are in `manuscripts/`.
 

pen_stack-6.7.0/benchmarks/position_effect/README.md

@@ -0,0 +1,60 @@
+# TPE-Bench — the Position-Effect / Expression track
+
+The **expression** capability of PEN-STACK never had a held-out benchmark. TPE-Bench is the position-effect
+track for the [Genome-Writing Challenge](../genome_writing_challenge/) (v6.7 PEN-EXPRESS): given a genomic
+**chromatin context** + **cassette**, predict the **integrated-reporter expression** (and silencing) — scored on
+a **sealed held-out split** whose labels the submitter never sees.
+
+## Why
+
+Position-effect (where an integrated cassette lands → how strongly/durably it expresses) is the writing-relevant
+quantity no safe-harbour resource predicts. TRIP (Akhtar 2013) supervises it directly. TPE-Bench seals a held-out
+split and anchors a baseline leaderboard, so others can build *to* a calibrated expression predictor.
+
+## Tracks
+
+| Track | Status | What is held out | Metric |
+|---|---|---|---|
+| `chrom_holdout` | **LIVE** | whole chromosomes (`chr2, chr5, chr14, chrX`), frozen + SHA-locked in `split.json` | Spearman ρ (expression) + AUROC (silenced) |
+| `celltype_holdout` | **DATA-GATED** | leave-one-cell-type-out (the headline transfer track) | — |
+
+`celltype_holdout` is the headline cross-cell-type transfer test. With a single available position-effect cell
+type (mESC) it reports `data_gated` **honestly** and activates once PatchMPRA / MPIRE / lentiMPRA / Leemans are
+fetched — **no transfer number is fabricated** until then.
+
+## Baseline leaderboard (sealed `chrom_holdout`, n_test = 2257)
+
+| Predictor | Expression ρ | Silenced AUROC |
+|---|---|---|
+| cassette-only (f_cassette) | 0.178 | — |
+| context-only (v3.x durability head) | 0.431 | 0.660 |
+| **PEN-EXPRESS factored (f_cassette + g_context)** | **0.475** | 0.660 |
+
+The factored model's gain is on **expression** (the cassette term lifts ρ 0.431 → 0.475 on the sealed test); the
+silenced classifier matches the durability head (the silencing question is chromatin-driven — reported honestly,
+not inflated). See `../../out/position_effect_report.json` for the full CV report + bootstrap CIs.
+
+## How to submit
+
+```python
+from benchmarks.position_effect.harness import Submission, evaluate
+
+def my_predict(public_input: dict):
+    # public_input = {task_id, family, cassette, chromatin_features:{H3K27ac,...}, instructions}; label hidden
+    return {"expression": 0.0, "p_silenced": 0.5}   # return your prediction (abstain-safe)
+
+print(evaluate(Submission(name="my-model", predict_fn=my_predict)))
+```
+
+Reference baselines: `python -c "from benchmarks.position_effect.harness import baseline_leaderboard as b; print(b())"`.
+
+## Rules
+
+- **Sealed + SHA-locked.** `split.json` (held-out chromosomes) is frozen and checksummed (`SHA256SUMS`) before
+  model selection — verify with `sha256sum -c SHA256SUMS`.
+- **No circular labels.** The label is the **measured** TRIP expression, never a submitter claim.
+- **Leakage-controlled.** Held out by whole chromosome (nearby integrations share chromatin).
+- **Honest data-gating.** The transfer track abstains until ≥2 cell types exist; no fabricated number.
+- **Deterministic.** PEN-EXPRESS anchors the leaderboard.
+
+Data: TRIP (Akhtar et al., *Cell* 2013; GEO GSE49806/GSE49807; trip.nki.nl). License: see `DATA_LICENSES.md`.

pen_stack-6.7.0/benchmarks/position_effect/SHA256SUMS

@@ -0,0 +1 @@
+2fc12cbf28531e68f6e25586da1d84003eeb6a098c9015998ce03f518546bc8a  split.json

{pen_stack-6.5.0 → pen_stack-6.7.0}/configs/genotoxicity_oracle.yaml

@@ -1,17 +1,17 @@
 version: '1.0'
-built: '2026-06-10'
+built: '2026-06-16'
 description: 'computed integration-site genotoxicity oracle: per vector class, the
-  observed enrichment of integration sites within window_bp of a COSMIC oncogene vs
-  genome background. genotox_score = min(1, 1/enrichment). In-vivo clonal outcome
-  is NOT modelled (stays a known-unknown).'
+  observed enrichment of integration sites within window_bp of a CancerMine (CC0)
+  oncogene vs genome background. genotox_score = min(1, 1/enrichment). In-vivo clonal
+  outcome is NOT modelled (stays a known-unknown).'
 window_bp: 50000
-genome_background_frac_oncogene_50kb: 0.02211
+genome_background_frac_oncogene_50kb: 0.11538
 inputs:
   visdb: VISDB per-virus hg38 catalogues
-  oncogenes: COSMIC CGC v104 (safety_annot)
+  oncogenes: CancerMine (CC0) via safety_annot
 provenance_dois:
 - 10.1093/nar/gkz867
-- 10.1038/s41568-018-0060-1
+- 10.1038/s41592-019-0422-y
 - 10.1016/S0092-8674(02)00864-4
 - 10.1126/science.1083413
 robust_min_n: 1000
@@ -19,29 +19,29 @@ classes:
   lentiviral:
     virus: HIV
     n_sites: 88743
-    frac_oncogene_50kb: 0.04602
-    ci95: 0.00138
-    enrichment: 2.081
+    frac_oncogene_50kb: 0.2244
+    ci95: 0.00274
+    enrichment: 1.945
     frac_genotoxic_cis: 0.000293
-    median_dist_oncogene: 2339098
+    median_dist_oncogene: 256384
     robust: true
   deltaretroviral:
     virus: HTLV
     n_sites: 51508
-    frac_oncogene_50kb: 0.02648
-    ci95: 0.00139
-    enrichment: 1.198
+    frac_oncogene_50kb: 0.14472
+    ci95: 0.00304
+    enrichment: 1.254
     frac_genotoxic_cis: 0.000369
-    median_dist_oncogene: 3766674
+    median_dist_oncogene: 547160
     robust: true
   gammaretroviral:
     virus: MLV
     n_sites: 32
-    frac_oncogene_50kb: 0.125
-    ci95: 0.11459
-    enrichment: 5.653
+    frac_oncogene_50kb: 0.21875
+    ci95: 0.14324
+    enrichment: 1.896
     frac_genotoxic_cis: 0.0
-    median_dist_oncogene: 2871705
+    median_dist_oncogene: 145675
     robust: false
 vehicle_class:
   lentiviral:

{pen_stack-6.5.0 → pen_stack-6.7.0}/configs/metric_guide.yaml

@@ -14,7 +14,7 @@ metrics:
     means: "relative insertional-oncogenesis safety of the delivery/integration strategy. 1.0 = episomal/
       non-integrating (no integration → no insertional mutagenesis); lower = an integrating vector whose insertion
       sites fall nearer known proto-oncogenes."
-    computed: "for integrating vectors, from VISDB integration-site maps × COSMIC Cancer-Gene-Census oncogene
+    computed: "for integrating vectors, from VISDB integration-site maps × CancerMine (CC0) oncogene
       proximity (per-vector enrichment of integrations within 50 kb of an oncogene vs background)."
     validation: "mechanistic proxy — NOT outcome-validated (the actual clonal-transformation rate is measured, not predicted)."
     reference: "e.g. lentiviral ~0.48 (2.08× oncogene-proximity enrichment) vs episomal AAV 1.0."

{pen_stack-6.5.0 → pen_stack-6.7.0}/configs/oracles/scope_cards.yaml

@@ -115,16 +115,16 @@ oracles:
 
   delivery_genotoxicity:        # v5.2 WS-GENOTOX: computed integration-site oncogene-proximity
     family: genome
-    version: "visdb+cgc_v104-2026"
+    version: "visdb+cancermine-2026"
     output_kind: baseline       # an observed-data comparator (integration catalogues), not generative
     valid_for: "RELATIVE genotoxicity ordering of INTEGRATING vector classes via the observed enrichment of
-      integration sites near COSMIC oncogenes (lentiviral vs gammaretroviral, from VISDB x CGC); reproduces the
-      lentivirus-safer-than-gammaretrovirus ordering from data"
+      integration sites near CancerMine (CC0) oncogenes (lentiviral vs gammaretroviral, from VISDB x CancerMine);
+      reproduces the integrating-vector-enriched-near-oncogenes signal from data"
     not_valid_for: "the IN-VIVO clonal-expansion / leukemogenesis OUTCOME in a patient (a known-unknown); an
       absolute per-insertion oncogenesis probability; non-integrating vectors (no insertional mechanism);
       classes with too few catalogued sites (flagged extrapolating)"
     generalizes_to_unseen_loci: false
-    license: "open (this work; VISDB 10.1093/nar/gkz867, COSMIC CGC 10.1038/s41568-018-0060-1)"
+    license: "open / CC0 (this work; VISDB 10.1093/nar/gkz867, CancerMine 10.1038/s41592-019-0422-y)"
 
   capsid_epitope:               # v5.3 WS-EPITOPE: computed capsid/envelope CD8 T-cell epitope load
     family: protein_design

pen_stack-6.7.0/docs/cards/position_effect_data.md

@@ -0,0 +1,36 @@
+# Data card — position-effect supervision (v6.7 PEN-EXPRESS)
+
+The unified table behind Stage H (`pen_stack/twin/data/position_effect.py`). One row = one integrated reporter /
+element measurement. Schema: `dataset, organism, cell_type, chrom, pos, cassette, expression_raw, expression_z,
+silenced, <chromatin features>`. `expression_z` is z-scored within (dataset × cassette).
+
+## Dataset registry (verified accessions — 2026-06-19 verification pass)
+
+| Dataset | Citation | DOI | Accession | Cell types | Status in v6.7 |
+|---|---|---|---|---|---|
+| **TRIP** | Akhtar et al., *Cell* 2013 | 10.1016/j.cell.2013.07.018 | GEO **GSE49806** (tetO) + **GSE49807** (mPGK); trip.nki.nl | mESC | **LIVE** (n=11,433) |
+| PatchMPRA | Maricque, Chaudhari & Cohen, *Nat Biotechnol* 2019 | 10.1038/nbt.4285 | GEO (per paper) | mESC | registered, data-gated |
+| MPIRE | Hong et al., *Nat Commun* 2024 | 10.1038/s41467-024-52599-6 | GEO **GSE223403**; github.com/claricehong/MPIRE_insulators | K562 | registered, data-gated |
+| lentiMPRA | Agarwal et al., *Nature* **639**:411–420 (2025) | 10.1038/s41586-024-08430-9 | ENCODE + GEO; bioRxiv 2023.03.05.531189 | HepG2, K562, WTC11 | registered, data-gated |
+| Leemans | Leemans et al., *Cell* 2019 | 10.1016/j.cell.2019.03.009 | GEO (per paper); van Steensel lab | K562 | registered, data-gated |
+
+> The bioRxiv id `2023.03.05.531189` is **lentiMPRA's** (Agarwal → *Nature* 2025), not e2MPRA/ccMPRA — corrected
+> in the 2026-06-19 citation verification pass.
+
+## TRIP (the LIVE supervision)
+
+- **What:** thousands of identical reporters integrated in parallel across the mESC genome; each row is a genomic
+  position with normalized expression → the position effect on an integrated cassette (the writing-relevant quantity).
+- **Columns used:** `chrom, pos, promoter` (cassette: tetO 10,903 / mPGK 530), `expression` (log2, −13…+9),
+  `silenced` (low-expression tail, 25%), + 5 chromatin marks (H3K27ac, H3K4me1, H3K4me3, H3K9me3, H3K27me3).
+- **Provenance:** same `trip_with_chromatin.parquet` the v3.x durability head trains on (regenerated by
+  `pen_stack/data/ingest_trip.py` + chromatin extraction).
+- **License:** TRIP data is from a public GEO deposit (Akhtar 2013); see `DATA_LICENSES.md`. Raw data is gitignored
+  (`data/external/`), pulled from the VM for local runs; the shipped wheel carries the loaders + accessions, not the data.
+
+## Honest limits
+
+- **Mouse, single cell type.** TRIP is mESC. The model learns `chromatin → expression` (never a coordinate), so it
+  *applies* to a human epigenome — but the cross-cell-type **transfer** is unproven until ≥2 cell types are unified.
+  Until then the transfer track is **data-gated** (no fabricated number).
+- **Reporter-based** (GFP-class), relative not absolute. Titer / % of normal stay known-unknowns.

pen_stack-6.7.0/docs/position_effect.md

@@ -0,0 +1,57 @@
+# Stage H — the learned, trained-conformal position-effect model (v6.7 PEN-EXPRESS)
+
+Stage H predicts how strongly an integrated cassette expresses in its **chromatin context** (the position effect).
+Through v6.6 this was a closed-form **heuristic** with a heuristic ±0.20 band that the code itself labelled *"NOT
+a trained conformal interval"*, and which **failed** independent validation (ρ=0.12 vs Damdindorj 2014). v6.7
+replaces the model behind Stage H with a **learned, decomposable, trained-conformal** one — wrapping the digital
+twin, not rebuilding it.
+
+## The model
+
+`pen_stack/twin/position_effect.py::PositionEffectModel` is **factored and decomposable**:
+
+```
+E_raw  ≈  f_cassette(cassette)            # the cassette's intrinsic strength (per-cassette mean)
+       +  g_context(chromatin features)    # the position effect — a LightGBM on local chromatin
+       (+ h_interaction, reported)         # does the context function differ by cassette? (separability)
+```
+
+`g_context` is supervised on the residual `E_raw − f_cassette`, so it learns the *position* effect on a scale
+comparable across cassettes. A `silenced` classifier shares the chromatin features. The model is wrapped with the
+**existing** `wgenome.uncertainty.ConformalRegressor` (chromosome-Mondrian split-conformal) and `wgenome.ood.OODDetector`
+— so a prediction is a **calibrated interval that widens out of distribution**.
+
+## Results (real, on TRIP supervision — Akhtar 2013, GEO GSE49806/49807, mESC, n=11,433)
+
+Chromosome-blocked GroupKFold; paired bootstrap 95% CIs. *Every number is from a real CV run — no fabrication.*
+
+| Metric | cassette-only | context-only (v3.x durability head) | **PEN-EXPRESS factored** | Δ vs head (CI) |
+|---|---|---|---|---|
+| Expression Spearman ρ | 0.032 | 0.427 | **0.469** | +0.041 [0.036, 0.046] ✅ excludes 0 |
+| Silenced AUROC | — | 0.647 | 0.651 | +0.004 [0.001, 0.007] ✅ excludes 0 |
+
+- **Gate G-M passed:** the factored model beats the durability head (CI excludes 0) → it serves behind Stage H.
+- **Separability:** interaction adds **−0.002** R² → *additive `f_cassette + g_context` suffices at this N*
+  (reported honestly; the cassette term lifts expression, the silencing question is chromatin-driven).
+- **Trained-conformal (the named gap, closed):** split-conformal (α=0.10) → **held-out coverage 0.885 vs 0.90
+  nominal** (within tolerance), qhat=5.50 on the log2 scale. Coverage is measured on a **half-chromosome held-out**
+  split, not on the calibration set.
+
+## Stage H integration (`twin/outcome.py`)
+
+`predict_outcome(design, cell_state)` now:
+- **With a chromatin context** (`design["chromatin_features"]`) **and the model artifact present** → serves the
+  learned **trained-conformal** interval + `p_silenced` + OOD tier in a `position_effect` block; `stage_h_mode =
+  "learned_trained_conformal"`.
+- **Without a context (or artifact)** → the closed-form heuristic band, exactly as before — **backward compatible**
+  (the v5.9 relative-scale contract and all prior tests are intact).
+
+## Honest limits
+
+- **Single-context supervision.** TRIP is mESC. The cross-cell-type **transfer** claim is **data-gated** — see
+  [tpe_bench.md](tpe_bench.md); no transfer number is fabricated until PatchMPRA/MPIRE/lentiMPRA/Leemans are fetched.
+- **Public data cannot earn the ✅** (the v6.5 wall). v6.7 ships the learned+calibrated upgrade + the benchmark, not
+  a manufactured validated axis. Titer / absolute expression / phenotype stay **known-unknowns**.
+- The model artifact (`models/position_effect.pkl`) is gitignored; regenerate it with
+  `python scripts/p1_build_position_effect.py` (the shipped calibration `configs/twin/position_effect_conformal.json`
+  is committed). Without the artifact, Stage H falls back to the heuristic.

pen_stack-6.7.0/docs/tpe_bench.md

@@ -0,0 +1,48 @@
+# TPE-Bench — the position-effect / expression benchmark (v6.7)
+
+The **expression** capability never had a held-out benchmark. TPE-Bench fills that gap as a track of the
+[Genome-Writing Challenge](../benchmarks/genome_writing_challenge/): given a genomic **chromatin context** +
+**cassette**, predict the **integrated-reporter expression** (and silencing), scored on a **sealed** split.
+
+Location: `benchmarks/position_effect/` (`harness.py`, `split.json`, `SHA256SUMS`, `README.md`).
+
+## Two tracks
+
+| Track | Status | Held out | Metric |
+|---|---|---|---|
+| `chrom_holdout` | **LIVE** | whole chromosomes `chr2, chr5, chr14, chrX` (frozen + SHA-locked) | Spearman ρ + AUROC |
+| `celltype_holdout` | **DATA-GATED** | leave-one-cell-type-out (the headline transfer test) | — |
+
+`celltype_holdout` is the cross-cell-type transfer test. With one available position-effect cell type (mESC) it
+returns `data_gated` **honestly** and activates once PatchMPRA / MPIRE / lentiMPRA / Leemans are fetched — **no
+transfer number is fabricated**.
+
+## Baseline leaderboard (`chrom_holdout`, sealed, n_test = 2257)
+
+| Predictor | Expression ρ | Silenced AUROC |
+|---|---|---|
+| cassette-only | 0.178 | — |
+| context-only (v3.x durability head) | 0.431 | 0.660 |
+| **PEN-EXPRESS factored** | **0.475** | 0.660 |
+
+The factored model's gain is on **expression** (ρ 0.431 → 0.475 on the sealed held-out chromosomes); the silencing
+classifier matches the durability head (chromatin-driven) — reported honestly, not inflated.
+
+## Discipline
+
+- **Sealed + SHA-locked.** `split.json` is frozen and checksummed (`SHA256SUMS`) **before model selection**;
+  verify with `sha256sum -c benchmarks/position_effect/SHA256SUMS`.
+- **Non-circular labels.** The label is the **measured** TRIP expression, never a submitter claim.
+- **Leakage-controlled.** Held out by whole chromosome (nearby integrations share chromatin).
+- **Honest data-gating.** The transfer track abstains until ≥2 cell types exist.
+
+## Submit
+
+```python
+from benchmarks.position_effect.harness import Submission, evaluate
+def predict(pi):  # pi = {task_id, cassette, chromatin_features, instructions}; label hidden
+    return {"expression": 0.0, "p_silenced": 0.5}
+print(evaluate(Submission("my-model", predict)))
+```
+
+Reproduce the baselines: `python -c "from benchmarks.position_effect.harness import baseline_leaderboard as b; print(b())"`.

{pen_stack-6.5.0 → pen_stack-6.7.0}/pen_stack/__init__.py

@@ -1,2 +1,2 @@
 """PEN-STACK v3.0 - open infrastructure for genome writing."""
-__version__ = "6.5.0"
+__version__ = "6.7.0"

{pen_stack-6.5.0 → pen_stack-6.7.0}/pen_stack/data/ingest_safety_annot.py

@@ -1,7 +1,7 @@
-"""Safety annotations per 1 kb bin (Phase 1, Step 1.4).
+"""Safety annotations per 1 kb bin (Phase 1, Step 1.4; v6.6 license-clean source).
 
-Builds per-bin safety features from COSMIC Cancer Gene Census (oncogene/TSG loci),
-DepMap CRISPRGeneEffect (essential genes), and GENCODE (gene/TSS distances):
+Builds per-bin safety features from CancerMine (CC0; oncogene/TSG loci — the default, shipped) or COSMIC CGC
+(local-only, bring-your-own-license), DepMap CRISPRGeneEffect (essential genes), and GENCODE (gene/TSS distances):
   - dist_oncogene, dist_tsg, dist_essential, dist_tss  (bp to nearest, via bedtools closest)
   - genotoxic_cis flag (bins within a window of LMO2/MECOM/CCND2/PRDM16/HMGA2)
 
@@ -48,6 +48,30 @@ def load_cosmic(tsv: str) -> pd.DataFrame:
     return df[["chrom", "start", "end", "GENE_SYMBOL", "role"]]
 
 
+# v6.6: CancerMine (CC0) is the license-clean oncogene/TSG/driver source that REPLACES COSMIC CGC in the shipped
+# artifact. Lever et al., Nat Methods 16:505-507 (2019), doi:10.1038/s41592-019-0422-y, CC0; Zenodo record 7689627.
+# COSMIC stays available (load_cosmic) but OFF by default, for local enrichment under the user's own license (BYO).
+CANCERMINE_URL = "https://zenodo.org/records/7689627/files/cancermine_collated.tsv?download=1"
+_ROLE_MAP = {"Oncogene": "oncogene", "Tumor_Suppressor": "TSG", "Driver": "driver"}
+
+
+def load_cancermine(tsv: str, genes: pd.DataFrame, min_citations: int = 3) -> pd.DataFrame:
+    """CC0 oncogene/TSG/driver list. The collated file has one row per (gene, cancer, role) with a citation_count;
+    we aggregate per gene-role across cancers (sum citations), keep roles with >= min_citations, and map the HUGO
+    symbol -> genomic coordinates via GENCODE. Returns the SAME schema as load_cosmic (chrom,start,end,GENE_SYMBOL,
+    role) with the role string containing 'oncogene'/'TSG'/'driver' so the downstream filters are unchanged."""
+    cm = pd.read_csv(tsv, sep="\t", dtype=str)
+    cm["cites"] = pd.to_numeric(cm.get("citation_count"), errors="coerce").fillna(0)
+    agg = cm.groupby(["gene_normalized", "role"], as_index=False)["cites"].sum()
+    agg = agg[agg["cites"] >= float(min_citations)]
+    agg["rn"] = agg["role"].map(_ROLE_MAP).fillna(agg["role"])
+    roles = (agg.groupby("gene_normalized")["rn"]
+             .apply(lambda s: ",".join(sorted(set(s)))).reset_index(name="role"))
+    g = genes[["gene_name", "chrom", "start", "end"]].drop_duplicates("gene_name")
+    m = roles.merge(g, left_on="gene_normalized", right_on="gene_name", how="inner")
+    return m.rename(columns={"gene_normalized": "GENE_SYMBOL"})[["chrom", "start", "end", "GENE_SYMBOL", "role"]]
+
+
 def load_depmap_essential(csv: str, thresh: float = -0.5) -> set[str]:
     """Common-essential genes: mean Chronos effect across cell lines < thresh."""
     df = pd.read_csv(csv, index_col=0)
@@ -112,17 +136,24 @@ def nearest_dist(bins_bed: pybedtools.BedTool, feat_df: pd.DataFrame, name: str)
     return out.groupby(["chrom", "start"], as_index=False)[name].min()
 
 
-def build(bin_grid: str, cosmic_tsv: str, depmap_csv: str, gencode_dest: str,
-          sizes_tsv: str, out_parquet: str) -> pd.DataFrame:
+def build(bin_grid: str, depmap_csv: str, gencode_dest: str, sizes_tsv: str, out_parquet: str, *,
+          source: str = "cancermine", cancermine_tsv: str | None = None, cosmic_tsv: str | None = None,
+          min_citations: int = 3) -> pd.DataFrame:
+    """Build per-bin safety features. `source` selects the LICENSE-CLEAN oncogene/TSG list: 'cancermine' (CC0,
+    default, shipped) or 'cosmic' (local-only, under the user's own license — bring-your-own-license enrichment)."""
     grid = pd.read_parquet(bin_grid)[["chrom", "start", "bin"]]
     bins_bed = _bed(grid.assign(end=grid["start"] + BIN_BP)).sort()
 
-    cosmic = load_cosmic(cosmic_tsv)
-    onco = cosmic[cosmic["role"].str.contains("oncogene", case=False, na=False)]
-    tsg = cosmic[cosmic["role"].str.contains("TSG", case=False, na=False)]
-
     gtf = download_gencode(gencode_dest)
     genes = parse_gencode_genes(gtf)
+
+    if source == "cosmic":
+        cg = load_cosmic(cosmic_tsv)
+    else:  # default: CancerMine (CC0)
+        cg = load_cancermine(cancermine_tsv, genes, min_citations=min_citations)
+    onco = cg[cg["role"].str.contains("oncogene", case=False, na=False)]
+    tsg = cg[cg["role"].str.contains("TSG", case=False, na=False)]
+
     ess_syms = load_depmap_essential(depmap_csv)
     ess = genes[genes["gene_name"].isin(ess_syms)]
 
@@ -148,15 +179,21 @@ def build(bin_grid: str, cosmic_tsv: str, depmap_csv: str, gencode_dest: str,
 def main() -> None:
     ap = argparse.ArgumentParser()
     ap.add_argument("--bin-grid", default="/data/features/bin_grid_1kb.parquet")
+    ap.add_argument("--source", choices=["cancermine", "cosmic"], default="cancermine",
+                    help="oncogene/TSG source: cancermine (CC0, shipped default) or cosmic (local-only BYO-license)")
+    ap.add_argument("--cancermine", default="/data/external/cancermine_collated.tsv")
+    ap.add_argument("--min-citations", type=int, default=3,
+                    help="CancerMine per-gene-role citation threshold (3 = validated precision: safety AUROC 0.74)")
     ap.add_argument("--cosmic", default="/data/external/Cosmic_CancerGeneCensus_v104_GRCh38.tsv")
     ap.add_argument("--depmap", default="/data/external/CRISPRGeneEffect.csv")
     ap.add_argument("--gencode", default="/data/raw/gencode.v46.basic.gtf.gz")
     ap.add_argument("--sizes", default="/data/raw/hg38.chrom.sizes")
     ap.add_argument("--out", default="/data/features/safety_annot.parquet")
     a = ap.parse_args()
-    df = build(a.bin_grid, a.cosmic, a.depmap, a.gencode, a.sizes, a.out)
+    df = build(a.bin_grid, a.depmap, a.gencode, a.sizes, a.out, source=a.source,
+               cancermine_tsv=a.cancermine, cosmic_tsv=a.cosmic, min_citations=a.min_citations)
     n_onco = (df["dist_oncogene"] == 0).sum()
-    print(f"safety_annot bins={len(df)} cols={[c for c in df.columns if c.startswith('dist') or c=='genotoxic_cis']}")
+    print(f"safety_annot[{a.source}] bins={len(df)} cols={[c for c in df.columns if c.startswith('dist') or c=='genotoxic_cis']}")
     print(f"bins in an oncogene={n_onco} genotoxic_cis bins={int(df['genotoxic_cis'].sum())}")