orfdata 0.4.0__tar.gz → 0.6.0__tar.gz

{orfdata-0.4.0 → orfdata-0.6.0}/MANIFEST.in

@@ -1,3 +1,3 @@
 include lint.sh test.sh deploy.sh
 recursive-include docs *.md
-recursive-include tests *.py *.json *.md
+recursive-include tests *.py *.json *.md *.tsv *.fasta

{orfdata-0.4.0/orfdata.egg-info → orfdata-0.6.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: orfdata
-Version: 0.4.0
+Version: 0.6.0
 Summary: Reference open reading frames and experimental translation evidence on annotated transcripts
 Author: OpenVax
 License-Expression: Apache-2.0
@@ -95,12 +95,16 @@ RNA products. Reference evidence stays separate from evidence for a variant or
 fusion. See the [API and integration guide](docs/reference.md).
 
 The catalogue API supports other species, with explicit species and assembly
-checks. **Bundled experimental evidence is currently human-only**. Offline tests
+checks. **The bundled experimental ORF catalogue is currently human-only**. Offline tests
 bind real human, mouse, dog and yeast reference slices; those fixtures are not
 new experimental ORF catalogues. [Species and reference identity](docs/species.md).
 
 Install with `pip install orfdata[pyensembl]`. See the
 [measurement API](docs/measurements.md) for native units, explicit samples and
-assays, and DataCache-managed downloads, and the
+assays, and DataCache-managed downloads. Downloadable
+[human, mouse and dog protein abundance](docs/pfizer-protein.md) uses the same
+species-scoped API. These canonical-search protein groups do not establish
+noncanonical ORF abundance. The
 [quantitative, multispecies data roadmap](https://github.com/openvax/orfdata/issues/3)
-for upcoming RNA abundance, ribosome occupancy and protein data releases.
+tracks RNA abundance, ribosome occupancy, yeast absolute quantification and
+additional protein data releases.

{orfdata-0.4.0 → orfdata-0.6.0}/README.md

@@ -68,12 +68,16 @@ RNA products. Reference evidence stays separate from evidence for a variant or
 fusion. See the [API and integration guide](docs/reference.md).
 
 The catalogue API supports other species, with explicit species and assembly
-checks. **Bundled experimental evidence is currently human-only**. Offline tests
+checks. **The bundled experimental ORF catalogue is currently human-only**. Offline tests
 bind real human, mouse, dog and yeast reference slices; those fixtures are not
 new experimental ORF catalogues. [Species and reference identity](docs/species.md).
 
 Install with `pip install orfdata[pyensembl]`. See the
 [measurement API](docs/measurements.md) for native units, explicit samples and
-assays, and DataCache-managed downloads, and the
+assays, and DataCache-managed downloads. Downloadable
+[human, mouse and dog protein abundance](docs/pfizer-protein.md) uses the same
+species-scoped API. These canonical-search protein groups do not establish
+noncanonical ORF abundance. The
 [quantitative, multispecies data roadmap](https://github.com/openvax/orfdata/issues/3)
-for upcoming RNA abundance, ribosome occupancy and protein data releases.
+tracks RNA abundance, ribosome occupancy, yeast absolute quantification and
+additional protein data releases.

{orfdata-0.4.0 → orfdata-0.6.0}/docs/measurements.md

@@ -6,8 +6,9 @@ proteins, protein groups and peptides. A measurement stays at the resolution
 the experiment supports. Parent-gene RNA abundance is useful context for an ORF;
 it is not a separate measurement of each ORF in that gene.
 
-Install `orfdata[data]` to manage datasets. The initial 0.3 registry is empty;
-the [data roadmap](https://github.com/openvax/orfdata/issues/3) tracks source
+Install `orfdata[data]` to manage datasets. The registry includes downloadable
+[Pfizer/Oziolor human, mouse and dog protein summaries](pfizer-protein.md);
+the [data roadmap](https://github.com/openvax/orfdata/issues/3) tracks further
 imports. The bundled human ORF evidence catalogue remains available offline.
 Its pooled PSM counts describe spectra, not sample-level protein abundance.
 
@@ -46,6 +47,10 @@ mapping table. The default PyEnsembl target uses its unversioned `id`; request
 mapping across annotation releases, gene isoforms, ORFs or species. A variant or
 fusion product does not inherit a native reference protein measurement.
 
+For native sources with unreported genomic metadata, use explicit
+[source references and mapping views](source-references.md) to retain the original
+observation alongside its mapping to a PyEnsembl target.
+
 The CLI uses the same operations:
 
 ```sh
@@ -96,7 +101,10 @@ An importer calls `QuantitativeDataset.create(path, manifest, samples=...,
 assays=..., measurements=..., mappings=...)`. These inputs may be generators;
 measurements are written incrementally. Every observation names a source asset
 or earlier input measurements. Derived inputs precede outputs in the same
-reference context. Duplicate IDs, broken foreign keys, wrong dataset releases
+dataset release. `input_measurements` names IDs within the output reference;
+`input_references` instead uses `MeasurementReference` records with each input's
+dataset, version, reference and ID. The two forms cannot be mixed. Cross-species
+derived abundance is rejected. Duplicate IDs, broken foreign keys, wrong dataset releases
 and cross-reference links fail the import. Failed builds do not replace data.
 
 Use `build_dataset(manifest, builder, ...)` for a managed local import.
@@ -123,6 +131,12 @@ definitions = {
 ```
 
 CLI users can supply this JSON with `orfdata datasets --registry FILE ...`.
+
+Rebuild supported releases from their pinned original inputs with
+`registry.build(name, version, source_paths={"asset_id": "/local/file"})` or
+`orfdata datasets build NAME --source-path ASSET=PATH`. Local overrides still
+require the published hashes. Rebuilds use a separate import cache and preserve
+installed releases. Unsupported source importers fail explicitly.
 `ORFDATA_CACHE`, then `OPENVAX_DATA_CACHE`, then the platform's OpenVax cache
 select the root. ORFdata owns its `orfdata/` child. Release cache identities
 include manifest/reference and content hashes. SQLite partitions every table

{orfdata-0.4.0 → orfdata-0.6.0}/docs/pairing.md

@@ -25,6 +25,13 @@ only explicitly concurrent assays are returned; opt into `longitudinal` or
 TMT channels and bridges remain in the assays and do not become biological
 sample counts.
 
+RNA annotation and protein search references can differ. `AssayPair.reference`
+is the left assay's context; `right_reference` explicitly identifies the right
+assay's context and defaults to the left for older records. Both must belong to
+the same species. A protein FASTA does not need an invented genomic assembly:
+see [source references](source-references.md). Original targets and references
+remain attached to both sides, even when their local IDs happen to match.
+
 The CLI streams the same table rows:
 
 ```sh
@@ -39,10 +46,13 @@ and cardinality. `same_target` compares one target across assays/samples.
 `parent_rna_context` links parent RNA to an ORF but keeps the RNA value attached
 to its original gene/transcript. `shared_group_member` records an ambiguous
 protein-group assignment; the group value stays a group value.
+`gene_protein_group_context` links RNA to an ambiguous source protein group
+without assigning that group's abundance to an individual protein or ORF.
 
 Importers pass `AssayPair` and `TargetRelation` iterators as `pairings=` and
 `relations=` to `QuantitativeDataset.create`. IDs belong to the dataset release
-and reference context. Invalid assay references, contradictory known donor
+and left reference context. The right assay/target keeps its own context.
+Invalid assay references, contradictory known donor
 identities and conflicting one-to-one assignments fail the import. Original
 storage-version-1 datasets remain readable and have no declared relationships.
 
@@ -67,7 +77,10 @@ assays. Missing, censored or nonpositive denominators produce an unresolved
 `not_measured` result with its reason, never infinity or a measured zero. RPF/RNA
 ratios are source-defined translation-efficiency measures, not measured flux.
 All derived values retain their input IDs, pairing, target attribution and
-normalization assumptions.
+normalization assumptions. When input reference contexts differ, the ratio
+stores `MeasurementReference` records in `input_references`, rather than
+unqualified local IDs. This preserves source identity across annotation and
+search-FASTA releases.
 
 ## Across species
 

orfdata-0.6.0/docs/pfizer-protein.md

@@ -0,0 +1,71 @@
+# Human, mouse and dog protein abundance
+
+The [Oziolor et al. atlas](https://doi.org/10.1038/s41597-026-07512-9) provides
+parallel TMT protein measurements. ORFdata keeps each species in a separate
+dataset, using the same samples, assays, measurements and query API.
+
+| Dataset | Reported protein groups | Source channels | Source rows |
+|---|---:|---:|---:|
+| `pfizer-atlas-human-protein` | 10,352 | 37 | 325,702 |
+| `pfizer-atlas-mouse-protein` | 12,124 | 63 | 615,991 |
+| `pfizer-atlas-dog-protein` | 9,929 | 38 | 313,263 |
+
+Release `2026-10-12.1` contains the complete March 23, 2026 summary tables from
+MassIVE MSV000099729, MSV000099754 and MSV000099744, respectively. Each source
+row contributes two observations: native `iBAQ` and `rIBAQ`. Neither is calibrated
+copies per cell. These are pooled tissue samples; rodent condition labels also
+include sex. Channels and technical runs do not count as independent donors.
+
+```sh
+pip install 'orfdata[data]'
+orfdata datasets list --species dog
+orfdata datasets download pfizer-atlas-dog-protein
+orfdata datasets query pfizer-atlas-dog-protein --quantity protein_abundance
+```
+
+```python
+from orfdata import DatasetRegistry, ScopedTarget
+
+registry = DatasetRegistry()
+dataset = registry.open("pfizer-atlas-dog-protein")  # After explicit download.
+reference, = dataset.manifest.references
+group = ScopedTarget(reference, "protein_group", "A0A0A0MPB0", "MSV000099744:ProteinGroup")
+for value in dataset.measurements(target=group):
+    print(value.sample_id, value.unit, value.value, value.status)
+```
+
+Every target retains its species and the checksum of the deposited UniProt
+search FASTA. The original genomic assembly and UniProt release are unknown.
+They stay unknown; a future annotation mapping must be explicit and validated.
+Group representatives can be inspected through `reference_mappings()`, but their
+lookup does not establish all group members. Opting into unresolved mapped
+views keeps the original group measurement visible, without assigning its value
+to an individual protein. No RNA pairings are inferred in this release.
+
+Source `NA` values remain missing and numeric zeros remain measured zeros.
+Absent group/channel combinations have no observation. For dog, 64,039 combinations
+are absent and eight reported rows contain `NA`, reproducing the existing
+64,047 missing iBAQ matrix cells. Each value retains its source line, column,
+original numeric text, representative, normalization sum and native `Abundance`
+field. Nonfinite transformed strings such as `-Inf` are kept as provenance only.
+
+These searches used UniProt sequences without spliced isoforms. Coverage does
+not establish that an unreported ncORF was searched or absent, nor does bulk
+tissue MS establish MHC presentation. The article and deposit disagree on the
+instrument name; that discrepancy remains explicit in assay metadata.
+
+Downloads are 87.3 MB (human), 154.8 MB (mouse) and 84.4 MB (dog). Installed
+SQLite files occupy 1.46 GB, 2.72 GB and 1.40 GB, respectively. Queries stream
+observations; they do not load the entire file or a dense matrix into memory.
+
+Source tables are CC0 according to the MassIVE deposits; UniProt search records
+retain UniProt Consortium attribution and CC-BY-4.0 terms. Both sources have
+pinned hashes and sizes. The data artifact is separate from the Python package.
+Discovery and installed queries work offline. `download` verifies and installs
+the compressed artifact; it does not fetch the large vendor search database.
+
+To reproduce a source import, use `registry.build(name, source_paths=...)`,
+`orfdata datasets build NAME`, or `orfdata.sources.pfizer.build_protein("dog")`.
+The source overrides are `protein.tsv` and `search.fasta`; DataCache verifies
+their original checksums before running the importer. Source slices with actual
+positive, zero and missing values test the same importer for all three species.

{orfdata-0.4.0 → orfdata-0.6.0}/docs/reference.md

@@ -75,9 +75,8 @@ reconstructed RNA products and preserves each candidate's RNA support. For
 example, the native TPST1 peptide KIILSKISV is absent from the Sid TPST1–CRCP
 alternatives, so its MS evidence does not support those altered products.
 
-The reference integration is available in Varcode 10.12.0; the Isovar 1.50.0
-integration is in review. The stricter reference-context contract is coordinated
-with Varcode 10.13.0 and Isovar 1.51.0.
+The reference integration, including strict reference-context validation, is
+available in Varcode 10.13.0 and Isovar 1.51.0.
 
 ## Data and limits
 

orfdata-0.6.0/docs/source-references.md

@@ -0,0 +1,90 @@
+# Source references and PyEnsembl mappings
+
+A proteomics experiment may search a known protein FASTA without naming a
+genomic assembly. An older RNA study may report its annotation provider but
+omit the release. ORFdata keeps these source identities and their unknown fields.
+
+```python
+from orfdata import ReferenceContext
+
+source = ReferenceContext(
+    taxon_id=9615, species="dog", genome_build=None,
+    annotation_name="uniprot", annotation_version=None,
+    sequence_sha256=searched_fasta_sha256,
+)
+```
+
+Unknown assembly/provider fields require a pinned `sequence_sha256` or
+`source_sha256`. The former identifies a sequence set; the latter can pin an
+input table when its reference release is unreported. These source contexts
+cannot bind a genomic transcript. A `ReferenceORF` still requires a complete
+genomic model validated against its actual PyEnsembl transcript. Existing
+complete-reference keys are unchanged.
+
+## Query through an explicit mapping
+
+`ReferenceMapping` connects a native `ScopedTarget` to all candidate targets in
+one destination reference. It records the dataset/version, mapping method,
+validation, source provenance, status and unresolved/unmapped reason. Importers
+must perform the stated checks against the pinned inputs. Unique identifier
+resolution is separate from sequence validation.
+
+```python
+from orfdata import ScopedTarget
+
+target = ScopedTarget.from_pyensembl(gene, kind="gene", namespace="ensembl:110")
+for view in dataset.mapped_measurements(target):
+    print(view.measurement.value, view.measurement.unit)
+    print(view.measurement.target, view.requested_target)
+    print(view.mapping.method, view.mapping.validation)
+```
+
+The view contains the **original observation**, including its source reference,
+target, sample, assay, unit and provenance. Nothing is relabeled or copied onto
+the requested target. Direct `dataset.measurements(target=...)` still returns
+only observations native to that exact target/reference.
+
+By default, mapped queries return uniquely resolved assignments. Use
+`mapping_statuses=["ambiguous", "unresolved"]` to inspect other candidates, and
+`require_sequence=True` to require recorded exact sequence validation. A group
+value returned for a candidate member has `attribution="group_membership_context"`;
+it does not independently quantify that protein. Parent gene/transcript RNA
+cannot map directly onto an ORF. Use a separate parent-context relationship.
+
+The CLI exposes the same query:
+
+```sh
+orfdata datasets query DATASET --mapped --reference ref:SHA256 \
+    --kind protein --namespace ensembl:110 --target PROTEIN_ID --require-sequence
+```
+
+## Importer contract
+
+Pass `reference_mappings=` to `QuantitativeDataset.create`. Declare source and
+destination contexts in the manifest. Each source/destination-reference pair
+has one mapping containing all candidates, preventing competing unique claims.
+Unmapped records remain visible through `dataset.reference_mappings()`.
+
+The status is `unique`, `ambiguous`, `unmapped` or `unresolved`. A single reported
+protein representative with incomplete group membership is unresolved; its
+measurement cannot become a uniquely measured member. Identifier-based mappings
+must say what was checked and leave sequence validation absent when unavailable.
+For an exact sequence search, record:
+
+```python
+validation = {"sequence": {
+    "source_sha256": source_protein_sequence_sha256,
+    "target_sha256": matching_protein_sequence_sha256,
+    "all_candidates_checked": True,
+}}
+```
+
+These are hashes of the compared sequences, distinct from the whole searched
+FASTA's hash in the reference context. Both hashes must agree to count as an
+exact match. ORF assignments additionally require their own `frame` and
+`position` evidence. Known reference-strain differences need explicit
+`strain_relationship` evidence. Cross-species mappings use the separate
+[orthology API](pairing.md#across-species).
+
+Storage version 3 preserves these mappings. Earlier versions remain readable
+and report no declared reference mappings; no mapping is inferred on load.

{orfdata-0.4.0 → orfdata-0.6.0}/orfdata/__init__.py

@@ -15,7 +15,15 @@ from .catalog import (
 from .dataset import QuantitativeDataset
 from .datasets import DatasetRegistry, build_dataset
 from .initiation_context import KOZAK_PREFERENCES, InitiationContext, annotate_initiation_context
-from .measurements import Assay, DatasetManifest, Measurement, Sample, SourceAsset, TargetMapping
+from .measurements import (
+    Assay,
+    DatasetManifest,
+    Measurement,
+    MeasurementReference,
+    Sample,
+    SourceAsset,
+    TargetMapping,
+)
 from .model import ORFModel, ReferenceORF
 from .pairing import (
     AssayPair,
@@ -26,6 +34,7 @@ from .pairing import (
     compare_orthologs,
 )
 from .reference import ReferenceContext, ScopedTarget
+from .reference_mapping import MappedMeasurement, ReferenceMapping
 from .version import __version__
 
 __all__ = [
@@ -47,6 +56,7 @@ __all__ = [
     "Sample",
     "Assay",
     "Measurement",
+    "MeasurementReference",
     "TargetMapping",
     "AssayPair",
     "TargetRelation",
@@ -54,6 +64,8 @@ __all__ = [
     "OrthologyMapping",
     "OrthologyComparison",
     "compare_orthologs",
+    "ReferenceMapping",
+    "MappedMeasurement",
     "load_catalog",
     "UORFCatalog",
     "UORFRecord",