orfdata 0.1.0__tar.gz → 0.3.0__tar.gz

orfdata-0.3.0/MANIFEST.in

@@ -0,0 +1,3 @@
+include lint.sh test.sh deploy.sh
+recursive-include docs *.md
+recursive-include tests *.py *.json *.md

{orfdata-0.1.0 → orfdata-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: orfdata
-Version: 0.1.0
+Version: 0.3.0
 Summary: Reference open reading frames and experimental translation evidence on annotated transcripts
 Author: OpenVax
 License-Expression: Apache-2.0
@@ -11,6 +11,10 @@ Provides-Extra: pyensembl
 Requires-Dist: pyensembl>=2.27.2; extra == "pyensembl"
 Provides-Extra: build-data
 Requires-Dist: openpyxl>=3.1; extra == "build-data"
+Provides-Extra: data
+Requires-Dist: datacache<2,>=1.25.1; extra == "data"
+Provides-Extra: columnar
+Requires-Dist: pyarrow>=14; extra == "columnar"
 Provides-Extra: dev
 Requires-Dist: pytest; extra == "dev"
 Requires-Dist: ruff; extra == "dev"
@@ -18,6 +22,7 @@ Requires-Dist: build; extra == "dev"
 Requires-Dist: twine; extra == "dev"
 Requires-Dist: openpyxl>=3.1; extra == "dev"
 Requires-Dist: pyensembl>=2.27.2; extra == "dev"
+Requires-Dist: datacache<2,>=1.25.1; extra == "dev"
 Dynamic: license-file
 
 # orfdata
@@ -90,9 +95,12 @@ RNA products. Reference evidence stays separate from evidence for a variant or
 fusion. See the [API and integration guide](docs/reference.md).
 
 The catalogue API supports other species, with explicit species and assembly
-checks. **Bundled evidence is currently human-only**; mouse bindings use synthetic
-test data. [Adding another species](docs/reference.md#other-species).
+checks. **Bundled experimental evidence is currently human-only**. Offline tests
+bind real human, mouse, dog and yeast reference slices; those fixtures are not
+new experimental ORF catalogues. [Species and reference identity](docs/species.md).
 
 Install with `pip install orfdata[pyensembl]`. See the
+[measurement API](docs/measurements.md) for native units, explicit samples and
+assays, and DataCache-managed downloads, and the
 [quantitative, multispecies data roadmap](https://github.com/openvax/orfdata/issues/3)
-for planned RNA abundance, ribosome occupancy and protein measurements.
+for upcoming RNA abundance, ribosome occupancy and protein data releases.

{orfdata-0.1.0 → orfdata-0.3.0}/README.md

@@ -68,9 +68,12 @@ RNA products. Reference evidence stays separate from evidence for a variant or
 fusion. See the [API and integration guide](docs/reference.md).
 
 The catalogue API supports other species, with explicit species and assembly
-checks. **Bundled evidence is currently human-only**; mouse bindings use synthetic
-test data. [Adding another species](docs/reference.md#other-species).
+checks. **Bundled experimental evidence is currently human-only**. Offline tests
+bind real human, mouse, dog and yeast reference slices; those fixtures are not
+new experimental ORF catalogues. [Species and reference identity](docs/species.md).
 
 Install with `pip install orfdata[pyensembl]`. See the
+[measurement API](docs/measurements.md) for native units, explicit samples and
+assays, and DataCache-managed downloads, and the
 [quantitative, multispecies data roadmap](https://github.com/openvax/orfdata/issues/3)
-for planned RNA abundance, ribosome occupancy and protein measurements.
+for upcoming RNA abundance, ribosome occupancy and protein data releases.

orfdata-0.3.0/deploy.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+# Publish only a reviewed version from clean, synchronized main.
+set -euo pipefail
+PYTHON="${PYTHON:-python}"
+[[ "$(git branch --show-current)" == main ]] || { echo 'Release from main only' >&2; exit 1; }
+[[ -z "$(git status --porcelain)" ]] || { echo 'Release requires a clean tree' >&2; exit 1; }
+git fetch origin main
+[[ "$(git rev-parse HEAD)" == "$(git rev-parse origin/main)" ]] || { echo 'Synchronize main first' >&2; exit 1; }
+version="$($PYTHON -c 'from orfdata.version import __version__; print(__version__)')"
+if git rev-parse "v$version" >/dev/null 2>&1; then echo 'Version tag already exists' >&2; exit 1; fi
+./lint.sh
+./test.sh
+"$PYTHON" -m build
+"$PYTHON" -m twine check "dist/orfdata-$version.tar.gz" "dist/orfdata-$version-py3-none-any.whl"
+if [[ "${1:-}" == --dry-run ]]; then exit 0; fi
+"$PYTHON" -m twine upload "dist/orfdata-$version.tar.gz" "dist/orfdata-$version-py3-none-any.whl"
+git tag "v$version"
+git push origin "v$version"

orfdata-0.3.0/docs/design.md

@@ -0,0 +1,84 @@
+# General ORF reference and evidence layer
+
+Status: architectural correction to the unmerged oncoref/Varcode/Isovar PRs.
+Working distribution name: orfdata. No release is authorized by this document.
+
+## Ownership
+
+PyEnsembl remains responsible for reference genomes, annotations, transcripts,
+exons and canonical proteins. A small separate library adds source-defined ORFs
+on those transcripts and their experimental evidence. oncoref remains a consumer
+for cancer-specific analysis. Neither Varcode nor Isovar owns reference datasets.
+PepData's immune-epitope/property scope is not a fit.
+
+The new library has a dependency-light data layer and a PyEnsembl-compatible
+binding API. It must not import Varcode, Isovar or oncoref. No new PyEnsembl core
+API is needed to bind a Transcript; avoid reverse dependencies or monkey-patching.
+
+## Public objects
+
+- ORFModel: immutable source ORF identity, assembly, species (NCBI taxon ID and
+  scientific name), translation table, parent transcript, spliced coding blocks,
+  protein sequence, ORF class. ORF class is distinct from transcript biotype.
+- ORFEvidence: ribosome observations and peptide-level MS observations with their
+  source, accession, mapping ambiguity, assay and quality. Native evidence does
+  not become evidence for a modified product; orthology is not direct evidence.
+- ORFRecord: model plus evidence, retaining legacy flat accessors for Isovar.
+- ReferenceORF: validated binding of a record to a PyEnsembl Transcript, retaining
+  the original transcript (including noncoding biotype), cDNA coding interval,
+  coding sequence and actual resolved annotation/version. Multiple ORFs per RNA
+  are normal. Annotation lookup never relabels the transcript protein-coding.
+- ORFCatalog: explicitly one species/assembly per catalogue. Query by transcript,
+  gene, locus, class or evidence. for_transcript(transcript) validates and returns
+  ReferenceORFs. Load custom versioned catalogues offline. The initial bundled
+  human data is identified as human-only; no implied mouse evidence.
+- Varcode ORFAnnotation: reference ORF plus effect and realized products. Exports
+  separately identify reference model/evidence and modifications, preserving
+  phase/splice alternatives. A private transcript-engine adapter permits reuse
+  of existing variant logic without exposing a fake transcript as the base ORF.
+- Isovar: attach those same annotations to RNA products with candidate-specific
+  RNA support, maintaining conditional translation and unresolved RNA polarity.
+
+## Generality and compatibility
+
+Support uORF, overlapping uORF, internal/out-of-frame, downstream and noncoding-
+transcript ORFs with the same model and evidence schema. Preserve source classes
+verbatim rather than equating them with Ensembl transcript biotypes. Complete
+stopped ORFs are the initial binding contract; partial models are rejected
+explicitly rather than given invented boundaries.
+
+A generic v1 schema requires species, assembly, translation table and source
+provenance. Legacy Isovar human data may be read only through its explicit known
+human/GRCh38 provenance. Keep legacy Isovar uorf_* imports/call shapes working.
+New generic APIs use orf terminology; defaults remain explicit about dataset scope.
+
+Multi-species path: one versioned catalogue per organism/assembly, importer input
+using the same model/evidence schema, optional organism-specific source adapters
+(e.g. OpenProt). Test a mouse noncoding transcript with real PyEnsembl objects,
+plus human coding transcripts, different frames, minus strand, splice junctions,
+assembly/species mismatches and unknown translation tables. Synthetic examples
+are identified as test fixtures, never scientific observations. OpenProt already
+publishes multiple species with Ensembl/RefSeq mappings and evidence filters.
+
+## Review corrections
+
+Remove the oncoref runtime dependency from both consumers and withdraw its ORF
+PR when the replacement is reviewable. Keep unrelated lint fixes independently.
+Review evidence retention for duplicate reference codons (retain all copies),
+explicit translation tables and noncoding parents; preserve all Sid alternatives.
+Document unsupported de novo initiation discovery and partial ORFs honestly.
+
+## Verification
+
+New library: existing evidence/context/builder tests, generic data round-trip,
+PyEnsembl binding and multi-species isolation, independently installed wheel.
+Varcode/Isovar: focused tests then required full lint/test scripts. Regenerate
+Sid outputs from the original packaged reads. Existing human evidence counts and
+native peptide attribution must remain unchanged. Do not claim release until CI,
+merge and PyPI publication actually complete; publication remains unapproved.
+
+Primary sources:
+- https://www.nature.com/articles/s41587-022-01369-0
+- https://www.nature.com/articles/s41586-026-10459-x
+- https://archive.openprot.org/p/help
+- https://github.com/openvax/pyensembl

orfdata-0.3.0/docs/measurements.md

@@ -0,0 +1,134 @@
+# RNA, ribosome and protein measurements
+
+ORFdata adds evidence to PyEnsembl's reference transcripts and proteins. Its
+quantitative layer stores measurements of genes, transcripts, reference ORFs,
+proteins, protein groups and peptides. A measurement stays at the resolution
+the experiment supports. Parent-gene RNA abundance is useful context for an ORF;
+it is not a separate measurement of each ORF in that gene.
+
+Install `orfdata[data]` to manage datasets. The initial 0.3 registry is empty;
+the [data roadmap](https://github.com/openvax/orfdata/issues/3) tracks source
+imports. The bundled human ORF evidence catalogue remains available offline.
+Its pooled PSM counts describe spectra, not sample-level protein abundance.
+
+## Discover, download and query
+
+```python
+from orfdata import DatasetRegistry, ScopedTarget
+
+registry = DatasetRegistry()  # Metadata only; no download.
+releases = registry.list(species="dog", assay="digest_ms")
+```
+
+For a registered release, use `registry.info(name, version)` for its immutable
+manifest, `registry.download(name, version)` to fetch it, and
+`registry.open(name, version)` to read it offline. `inspect` reports missing,
+available or invalid files without downloading or repairing them.
+`download_metadata` returns the artifact's access terms and file hashes/sizes.
+An invalid cache
+requires an explicit `download(..., force=True)`; old reader paths survive.
+Python download options such as timeout, retries and progress go to DataCache.
+
+```python
+# dataset = registry.open(name, version)
+# transcript is a PyEnsembl Transcript in the dataset's exact reference context.
+target = ScopedTarget.from_pyensembl(
+    transcript, kind="transcript", namespace="ensembl:110"
+)
+for measurement in dataset.measurements(target=target):
+    print(measurement.value, measurement.unit, measurement.status)
+```
+
+Use `ScopedTarget.from_orf(bound_reference_orf, namespace=source_release)` for a
+validated reference ORF. Namespaces and identifiers must match the source's
+mapping table. The default PyEnsembl target uses its unversioned `id`; request
+`versioned=True` when the source uses versioned identifiers. There is no implicit
+mapping across annotation releases, gene isoforms, ORFs or species. A variant or
+fusion product does not inherit a native reference protein measurement.
+
+The CLI uses the same operations:
+
+```sh
+orfdata datasets list --species yeast
+orfdata datasets info DATASET --version RELEASE
+orfdata datasets download DATASET --version RELEASE
+orfdata datasets inspect DATASET --version RELEASE
+orfdata datasets query DATASET --version RELEASE --quantity protein_abundance
+```
+
+Queries stream JSON lines. `--reference ref:SHA256` selects an exact context;
+target queries also require `--target`, `--kind` and `--namespace`. Unqualified
+sample/assay ID queries across multiple references are rejected. Existing
+catalogue commands such as `orfdata --gene TPST1` retain their behavior.
+
+## What each value means
+
+`Measurement` retains the dataset/version, reference-scoped target, sample and
+assay IDs, value, native unit, linear/log scale and log base, uncertainty and its
+definition, detection limit, status, origin and provenance. Measured zero,
+below detection, not measured, not mapped and missing are distinct. Censored or
+missing values use `None`; native sentinels can remain in provenance.
+
+RNA counts/TPM/FPKM, ribosome occupancy, source-defined translation efficiency,
+LFQ/iBAQ/TMT values, peptide intensity, PSM counts, copies per cell and turnover
+remain distinct. No conversion is automatic. Derived/predicted values name a
+method and retain source/input provenance. An estimate from calibration remains
+an estimate; an intensity/TPM ratio does not acquire molecule-ratio units.
+
+`Sample` records biological units, pools and source labels. `Assay` separately
+records technical replicates, runs, fractions, TMT plex/channel/bridge,
+normalization and calibration. A sample's strain/breed belongs in its attributes;
+it need not be the strain used to assemble its reference genome. `TargetMapping`
+retains all candidate assignments or an unmapped reason. A protein-group
+representative does not become a uniquely measured protein. Explicit sample
+pairing and orthology comparisons follow in
+[issue #6](https://github.com/openvax/orfdata/issues/6).
+
+## Importing a source
+
+Define a `DatasetManifest` with exact reference contexts, raw source URLs,
+SHA-256 hashes and byte sizes, citations, file-specific access/redistribution
+terms, searched/measurable coverage, processing software/version/parameters and
+row accounting. Unavailable metadata stay unknown. Article licensing does not
+substitute for the data files' terms.
+
+An importer calls `QuantitativeDataset.create(path, manifest, samples=...,
+assays=..., measurements=..., mappings=...)`. These inputs may be generators;
+measurements are written incrementally. Every observation names a source asset
+or earlier input measurements. Derived inputs precede outputs in the same
+reference context. Duplicate IDs, broken foreign keys, wrong dataset releases
+and cross-reference links fail the import. Failed builds do not replace data.
+
+Use `build_dataset(manifest, builder, ...)` for a managed local import.
+`builder(source_paths, output_path)` reads the verified inputs and creates that
+SQLite dataset. DataCache publishes it atomically with input/output checksums
+and a processing receipt. `source_paths={asset_id: local_file}` supports manually
+acquired sources. Importing locally does not grant redistribution rights.
+
+A `DatasetRegistry(definitions, cache_root=...)` registers releases as:
+
+```python
+definitions = {
+    name: {
+        "default_version": version,
+        "versions": {version: {
+            "manifest": manifest.to_dict(),
+            "distribution": {"access_terms": terms, "redistributable": False},
+            "assets": {"dataset.sqlite": {
+                "url": url, "sha256": sha256, "size": size, "decompress": False,
+            }},
+        }},
+    }
+}
+```
+
+CLI users can supply this JSON with `orfdata datasets --registry FILE ...`.
+`ORFDATA_CACHE`, then `OPENVAX_DATA_CACHE`, then the platform's OpenVax cache
+select the root. ORFdata owns its `orfdata/` child. Release cache identities
+include manifest/reference and content hashes. SQLite partitions every table
+by reference key, with composite foreign keys; human, mouse, dog, yeast and
+arbitrary other taxa use the same schema. The core API needs neither pandas nor
+PyArrow. Optional `orfdata[columnar]` enables `dataset.export_parquet(path)`;
+query columns accompany full measurement JSON, preserving exact values and
+provenance even when a floating-point analysis column cannot represent an
+integer exactly.

orfdata-0.3.0/docs/reference.md

@@ -0,0 +1,145 @@
+# Using orfdata
+
+See the [README](../README.md) for the purpose, ORF types and basic example.
+
+## Objects and queries
+
+`load_catalog()` loads the bundled reference data offline. Catalogue access uses
+only the Python standard library; the optional `pyensembl` extra supplies
+PyEnsembl. Binding uses a transcript you have already loaded.
+
+`catalog.for_transcript(transcript)` returns a `ReferenceORF` for each matching
+ORF. Each has:
+
+- `model`: source identity, ORF type, coordinates, species and protein sequence.
+- `evidence`: ribosome observations and peptide-level MS observations.
+- `transcript`: the original PyEnsembl transcript, including its original biotype.
+- `coding_sequence` and `protein_sequence`: checked against that transcript.
+
+Binding checks the assembly, available species information, transcript identity,
+full spliced coding path, protein sequence and terminal stop. A matching gene
+name alone is insufficient. Incompatible annotations raise an error.
+
+To inspect source records without loading a transcript:
+
+```python
+from orfdata import load_catalog
+
+catalog = load_catalog()
+records = catalog.query(gene="TPST1", evidence="ms")
+```
+
+## Evidence
+
+Use `orf.evidence.has_evidence("ribosome")`, `"shotgun"` (protease-digest MS),
+`"mhc"` (including human HLA), or `"ms"` (any supporting MS observation).
+`"hla"` selects human HLA observations specifically. Source observations whose
+MS assay type is unspecified remain labelled `"ms_unclassified"`.
+
+```python
+orf.evidence.has_evidence("ms", require_unique=True, require_reviewed=True)
+```
+
+This requires a supporting spectrum review and a single protein mapping
+**within that source's search scope**. It does not imply uniqueness across every
+possible protein. Inspect `orf.evidence.peptides` for peptide sequences, source
+accessions, matching intervals, mapping counts and review quality. Missing counts
+remain unknown; low or mixed spectrum reviews do not count as supporting MS.
+
+The distinction between translation, peptide detection and functional protein
+evidence follows the [TransCODE study](https://www.nature.com/articles/s41586-026-10459-x).
+Source evidence belongs to the exact reference ORF. It is not automatically
+evidence for a different isoform, a mutant protein, a fusion or another species.
+
+## Variants and RNA products
+
+Varcode keeps the reference object alongside its modification:
+
+```python
+from pyensembl import EnsemblRelease
+from orfdata import load_catalog
+from varcode import Variant, annotate_orf_variants
+
+genome = EnsemblRelease(95, species="human")  # already downloaded and indexed
+transcript = genome.transcript_by_id("ENST00000304842")
+orf, = load_catalog().for_transcript(transcript)
+variant = Variant("7", 66205486, "C", "G", genome=genome)
+annotation = annotate_orf_variants([variant], orf)
+assert annotation.reference is orf
+print(annotation.effect)  # P2A in the TPST1 upstream ORF
+result = annotation.to_dict()  # reference, evidence and altered products
+```
+
+Isovar's `annotate_sv_orf_evidence` attaches the same reference annotations to
+reconstructed RNA products and preserves each candidate's RNA support. For
+example, the native TPST1 peptide KIILSKISV is absent from the Sid TPST1–CRCP
+alternatives, so its MS evidence does not support those altered products.
+
+The reference integration is available in Varcode 10.12.0; the Isovar 1.50.0
+integration is in review. The stricter reference-context contract is coordinated
+with Varcode 10.13.0 and Isovar 1.51.0.
+
+## Data and limits
+
+The bundled human GRCh38 catalogue, dataset `2026-10-11.1`, draws on the
+[GENCODE Phase I reference](https://www.nature.com/articles/s41587-022-01369-0),
+[TransCODE author exports](https://www.nature.com/articles/s41586-026-10459-x),
+and the [community microprotein spectrum benchmark](https://www.nature.com/articles/s41467-025-68002-x).
+All 7,264 ORF paths and proteins were checked against Ensembl 101 GTF, coding RNA
+and noncoding RNA FASTAs. The 2,186 models with supporting MS observations use
+the packaged source-specific filters; this is not a count of proven functional
+proteins. Packaged manifests retain source accessions, checksums, licenses,
+processing rules and dataset version.
+
+The current binder accepts complete ORFs with a terminal stop, NCBI translation
+table 1, and ATG/CTG/TTG initiation codons. All bundled models start at ATG.
+Other genetic codes, initiation conventions and incomplete ORFs need additional
+support. Transcript-boundary contexts can be partial; initiation efficiency and
+effects on the main coding region are not established by these data.
+
+Circular-RNA translation, stop-codon readthrough and translation from new
+transcripts or fusions are not covered by this reference catalogue. Those need
+additional source data and, where necessary, model support. ORF discovery is
+outside the current API.
+
+`load_uorf_catalog()` preserves Isovar's upstream-only snapshot: 3,771 models,
+dataset `2026-10-08.2`. New code should use the broader `load_catalog()` API.
+
+## Other species
+
+[ReferenceContext and ScopedTarget](species.md) keep taxa, strains, assemblies
+and annotation versions separate. Custom genomes without species/taxonomy need
+an explicit context when binding; source-pinned contexts require an exact match.
+
+`load_catalog(path)` reads a custom versioned JSON catalogue. Each catalogue is
+scoped to one species and assembly; each model records its NCBI taxon, scientific
+name, assembly, translation table and source. Human evidence is never transferred
+to a mouse ortholog. Mouse bindings are tested with synthetic ORFs on PyEnsembl
+transcripts; there is no bundled mouse evidence.
+
+To build a catalogue, create `ORFModel` objects from source coordinates and
+sequences, attach source observations with `ORFRecord.from_model(model, evidence)`,
+and construct `ORFCatalog(records, metadata)`. Its `to_dict()` produces the
+`orfdata.catalog.v1` JSON representation. Genomic coding blocks are 1-based,
+inclusive, sorted by genomic position, and include the terminal stop codon.
+
+[OpenProt](https://archive.openprot.org/p/help) is a possible source for further species,
+with organism/assembly-specific models, transcript mappings and experimental
+evidence. A future importer must pin releases, preserve ambiguity and licensing,
+and validate coordinate conventions. No OpenProt importer is included yet.
+
+## Rebuild and verify
+
+From a development checkout with the PyEnsembl dependency installed:
+
+```sh
+python -m pip install -e '.[dev]'
+python -m orfdata.build --source-dir /path/to/sources --prepare-sources \
+    --all-orfs --output orfdata/data/orf-evidence
+./lint.sh
+./test.sh
+```
+
+Source preparation downloads pinned public inputs and needs Rscript to extract
+the author export. Ordinary queries need neither R nor spreadsheet libraries.
+See the [design notes](design.md) for the model and library boundaries.

orfdata-0.3.0/docs/species.md

@@ -0,0 +1,58 @@
+# Species and references
+
+ORFdata uses the same objects for human, mouse, dog, yeast and custom species.
+Every ORF states its NCBI species taxon, assembly and genetic code. A
+`ReferenceContext` also pins the annotation provider/version and can retain a
+versioned assembly accession, strain taxon/label and sequence-set SHA-256.
+
+```python
+from orfdata import ReferenceContext, ScopedTarget
+
+yeast = ReferenceContext(
+    taxon_id=4932, species="saccharomyces_cerevisiae",
+    genome_build="R64-1-1", assembly_accession="GCA_000146045.2",
+    annotation_name="ensembl", annotation_version="115",
+    strain="S288C", strain_taxon_id=559292,
+)
+target = ScopedTarget(yeast, "protein", "YFL039C", "ensembl:115")
+print(target.key)
+```
+
+Target keys include the reference, source namespace, kind and source identifier.
+Identical local IDs in different species, reference versions or datasets remain
+separate. Known aliases such as `canis_familiaris` and
+`canis_lupus_familiaris` resolve to dog taxon 9615 through a versioned mapping;
+the original name stays in exports. Assembly names are never automatically
+equated, and strain identity is never discarded. Unknown species require their
+own explicit taxonomy context; they do not require a schema change.
+
+`ReferenceContext.from_genome(genome)` resolves available PyEnsembl metadata
+offline. A custom Genome without species/taxonomy needs an explicit context:
+
+```python
+bound = record.bind(transcript, context=yeast)
+# Or: catalog.for_transcript(transcript, context=yeast)
+```
+
+The caller declares metadata absent from a custom genome; any metadata already
+present must agree. The binder still checks transcript version, strand, complete
+spliced path, coding sequence and protein. An ORF model with a
+`reference_context` requires that exact context. Older catalogues without a
+pinned annotation context can bind an identical transcript in another release;
+the actual resolved context and transcript/coding/protein hashes are exported.
+No binding downloads a reference or performs liftover.
+
+`context.validate_genome(genome)` checks available metadata against an explicit
+context. Genomic alleles can use `include_annotation=False`: their coordinates
+are independent of a gene annotation release, while taxonomy, strain and
+assembly checks still apply. This does not relax ORF or RNA-product binding.
+
+`ScopedTarget` supports genes, transcripts, ORFs, proteins, ambiguous protein
+groups and peptides. A parent's gene or RNA measurement remains at its original
+resolution; this identity layer does not copy it onto every ORF. Orthology and
+equal peptide sequences do not transfer observations between species.
+
+The binder currently supports complete stopped ORFs under NCBI table 1, with
+ATG/CTG/TTG initiation. Unsupported codes and initiation codons fail explicitly.
+In particular, yeast nuclear and mitochondrial codes differ; a contig named
+`MT` never selects a code implicitly. See [NCBI's code definitions](https://www.ncbi.nlm.nih.gov/Taxonomy/Utils/wprintgc.cgi).

orfdata-0.3.0/lint.sh

@@ -0,0 +1,3 @@
+#!/bin/sh
+set -eu
+"${PYTHON:-python}" -m ruff check orfdata tests

{orfdata-0.1.0 → orfdata-0.3.0}/orfdata/__init__.py

@@ -12,8 +12,12 @@ from .catalog import (
     load_catalog,
     load_uorf_catalog,
 )
+from .dataset import QuantitativeDataset
+from .datasets import DatasetRegistry, build_dataset
 from .initiation_context import KOZAK_PREFERENCES, InitiationContext, annotate_initiation_context
+from .measurements import Assay, DatasetManifest, Measurement, Sample, SourceAsset, TargetMapping
 from .model import ORFModel, ReferenceORF
+from .reference import ReferenceContext, ScopedTarget
 from .version import __version__
 
 __all__ = [
@@ -25,6 +29,17 @@ __all__ = [
     "PeptideEvidence",
     "ORFModel",
     "ReferenceORF",
+    "ReferenceContext",
+    "ScopedTarget",
+    "QuantitativeDataset",
+    "DatasetRegistry",
+    "build_dataset",
+    "DatasetManifest",
+    "SourceAsset",
+    "Sample",
+    "Assay",
+    "Measurement",
+    "TargetMapping",
     "load_catalog",
     "UORFCatalog",
     "UORFRecord",

{orfdata-0.1.0 → orfdata-0.3.0}/orfdata/catalog.py

@@ -15,6 +15,7 @@ from typing import Optional
 
 from .initiation_context import KOZAK_PREFERENCES, InitiationContext
 from .model import ORFModel, ReferenceORF, _species
+from .reference import ReferenceContext
 from .sequence import CODONS, reverse_complement_dna
 
 EVIDENCE_METHODS = ("ribosome", "shotgun", "mhc", "hla", "ms_unclassified", "ms")
@@ -114,6 +115,7 @@ class ORFRecord:
     species: Optional[str] = None
     translation_table: int = 1
     source: str = ""
+    reference_context: Optional[ReferenceContext] = None
 
     @property
     def model(self):
@@ -133,6 +135,7 @@ class ORFRecord:
             species=self.species,
             translation_table=self.translation_table,
             source=self.source,
+            reference_context=self.reference_context,
         )
 
     @property
@@ -144,6 +147,8 @@ class ORFRecord:
         """Construct a record from the generic model and independent evidence."""
         evidence = ORFEvidence() if evidence is None else evidence
         fields = model.to_dict()
+        if model.reference_context is not None:
+            fields["reference_context"] = model.reference_context
         fields["biotype"] = fields.pop("orf_type")
         return cls(
             **fields,
@@ -152,13 +157,15 @@ class ORFRecord:
             initiation_context=initiation_context,
         )
 
-    def bind(self, transcript):
+    def bind(self, transcript, *, context=None):
         """Validate and bind this ORF to a PyEnsembl transcript without relabelling it."""
-        return ReferenceORF(self, transcript)
+        return ReferenceORF(self, transcript, context=context)
 
     def to_dict(self):
         """Return a serializable copy of this record."""
         result = asdict(self)
+        if self.reference_context is None:
+            result.pop("reference_context")
         if self.initiation_context is not None:
             result["initiation_context"] = self.initiation_context.to_dict()
         return result
@@ -348,7 +355,7 @@ class ORFCatalog:
             if r.genome_build != self.genome_build or r.strand not in ("+", "-"):
                 raise ValueError(f"Invalid assembly or strand for {r.orf_id}")
             r.model  # Validate identity and geometry, including non-upstream ORFs.
-            if (r.taxon_id, r.species) != (self.taxon_id, self.species):
+            if (r.taxon_id, _species(r.species)) != (self.taxon_id, _species(self.species)):
                 raise ValueError("Catalogue and ORF species disagree")
             if (
                 not r.coding_blocks
@@ -425,7 +432,7 @@ class ORFCatalog:
         """Return an exact source ORF ID; unknown IDs raise KeyError."""
         return self._by_id[orf_id]
 
-    def for_transcript(self, transcript, **filters):
+    def for_transcript(self, transcript, *, context=None, **filters):
         """Return validated ORFs on this transcript, with optional evidence filters.
 
         A source model that disagrees with the selected annotation raises rather
@@ -436,8 +443,11 @@ class ORFCatalog:
         species = getattr(transcript.genome, "species", None)
         if species is not None and _species(species) != _species(self.species):
             raise ValueError("Catalogue transcript species mismatch")
+        resolved = ReferenceContext.from_genome(transcript.genome, context=context)
+        if resolved.taxon_id != self.taxon_id:
+            raise ValueError("Catalogue transcript taxon mismatch")
         return tuple(
-            record.bind(transcript)
+            record.bind(transcript, context=resolved)
             for record in self.query(**filters)
             if record.transcript_id.split(".")[0] == str(transcript.id).split(".")[0]
         )
@@ -576,6 +586,8 @@ def _load_catalog(path=None, *, expected_sha256=None, resource="orf-evidence"):
             )
         row["coding_blocks"] = tuple(tuple(x) for x in row["coding_blocks"])
         row["ribosome_studies"] = tuple(row["ribosome_studies"])
+        if row.get("reference_context") is not None:
+            row["reference_context"] = ReferenceContext(**row["reference_context"])
         if row.get("initiation_context") is not None:
             context = dict(row["initiation_context"])
             assessment = context.pop("assessment", None)