orfdata 0.9.0__tar.gz → 0.11.0__tar.gz

{orfdata-0.9.0 → orfdata-0.11.0}/MANIFEST.in

@@ -3,3 +3,4 @@ recursive-include docs *.md
 recursive-include tests *.py *.json *.md *.tsv *.fasta
 recursive-include tests/data/copy *.xlsx *.gz *.soft *.xml *.pdf
 recursive-include tests/data/storage *.sqlite
+recursive-include tests/data/pfizer_rna *.gz *.xlsx

{orfdata-0.9.0/orfdata.egg-info → orfdata-0.11.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: orfdata
-Version: 0.9.0
+Version: 0.11.0
 Summary: Reference open reading frames and experimental translation evidence on annotated transcripts
 Author: OpenVax
 License-Expression: Apache-2.0
@@ -104,8 +104,10 @@ Install with `pip install orfdata[pyensembl]`. See the
 assays, and DataCache-managed downloads. Downloadable
 [human, mouse and dog protein abundance](docs/pfizer-protein.md) and
 [yeast calibrated protein/RNA abundance](docs/copy-yeast.md) use the same
-species-scoped API. These canonical-search measurements do not establish
-noncanonical ORF abundance. The
+species-scoped API, as do [native dog and mouse RNA counts](docs/pfizer-rna.md).
+Gene RNA and canonical protein measurements do not establish
+noncanonical ORF abundance. The [native search references](docs/search-references.md)
+preserve complete deposited sequences, including unreported entries. The
 [quantitative, multispecies data roadmap](https://github.com/openvax/orfdata/issues/3)
 tracks further RNA abundance, ribosome occupancy, absolute quantification and
-additional protein data releases.
+additional protein data releases. See the [released coverage and gaps](docs/coverage.md).

{orfdata-0.9.0 → orfdata-0.11.0}/README.md

@@ -77,8 +77,10 @@ Install with `pip install orfdata[pyensembl]`. See the
 assays, and DataCache-managed downloads. Downloadable
 [human, mouse and dog protein abundance](docs/pfizer-protein.md) and
 [yeast calibrated protein/RNA abundance](docs/copy-yeast.md) use the same
-species-scoped API. These canonical-search measurements do not establish
-noncanonical ORF abundance. The
+species-scoped API, as do [native dog and mouse RNA counts](docs/pfizer-rna.md).
+Gene RNA and canonical protein measurements do not establish
+noncanonical ORF abundance. The [native search references](docs/search-references.md)
+preserve complete deposited sequences, including unreported entries. The
 [quantitative, multispecies data roadmap](https://github.com/openvax/orfdata/issues/3)
 tracks further RNA abundance, ribosome occupancy, absolute quantification and
-additional protein data releases.
+additional protein data releases. See the [released coverage and gaps](docs/coverage.md).

orfdata-0.11.0/docs/coverage.md

@@ -0,0 +1,34 @@
+# Released data and gaps
+
+ORFdata keeps datasets in separate species/reference contexts and exposes them
+through the same discovery, download and query API. This table describes actual
+data coverage, independently of what the API can represent.
+
+| Species | RNA abundance | Protein abundance | RNA/protein pairing |
+| --- | --- | --- | --- |
+| Human | Native Pfizer counts missing from the deposit | Pfizer: 10,352 groups, 37 channels, iBAQ/rIBAQ | Unresolved |
+| Mouse | Pfizer: 606 libraries, 56,051 gene targets, counts | Pfizer: 12,124 groups, 63 channels, iBAQ/rIBAQ | Unresolved library-to-pool correspondence |
+| Dog | Pfizer: 434 libraries, 30,941 gene targets, counts | Pfizer: 9,929 groups, 38 channels, iBAQ/rIBAQ | Unresolved library-to-pool correspondence |
+| Budding yeast | CoPY: four RNA libraries plus a source aggregate, FPKM and estimated copies/cell | CoPY: calibrated SRM and separately calibrated label-free estimates | Explicit source-cohort pairing; individual cross-assay replicate matches unknown |
+
+Gene targets include measured zeros; their presence in a matrix does not imply
+expression in every sample. Protein-group representatives do not establish
+unique protein attribution. TMT channels are not independent donors. CoPY RNA
+copies/cell assume 60,000 total mRNAs; its 1,903 SRM summaries contain 1,167 point
+estimates, 668 upper bounds and 68 unquantifiable results. Calibration and
+uncertainty remain attached to observations.
+
+The separate existence-evidence catalogue contains 7,264 human noncanonical
+ORFs, including 2,186 with reported MS support. It has no released mouse, dog or
+yeast experimental ncORF catalogue yet. Its ribosome evidence and pooled PSM
+counts are not sample-level ribosome occupancy or protein abundance. No species
+currently has a released quantitative Ribo-seq dataset in this registry.
+
+See the short guides for [protein measurements](pfizer-protein.md),
+[native dog/mouse RNA](pfizer-rna.md), [yeast RNA/protein calibration](copy-yeast.md)
+and [complete search sequences](search-references.md). The
+[roadmap](https://github.com/openvax/orfdata/issues/3) tracks matched human
+comparators, quantitative Ribo-seq, further ncORF evidence and additional species.
+Cat, cattle, rat and other taxa use the same model when source datasets are
+added. Orthology comparisons are explicit and never transfer observations or
+protein-existence evidence between species.

{orfdata-0.9.0 → orfdata-0.11.0}/docs/measurements.md

@@ -8,6 +8,8 @@ it is not a separate measurement of each ORF in that gene.
 
 Install `orfdata[data]` to manage datasets. The registry includes downloadable
 [Pfizer/Oziolor human, mouse and dog protein summaries](pfizer-protein.md);
+[native dog/mouse RNA](pfizer-rna.md) and [yeast RNA/protein calibration](copy-yeast.md)
+use the same objects. See [released coverage and gaps](coverage.md);
 the [data roadmap](https://github.com/openvax/orfdata/issues/3) tracks further
 imports. The bundled human ORF evidence catalogue remains available offline.
 Its pooled PSM counts describe spectra, not sample-level protein abundance.

{orfdata-0.9.0 → orfdata-0.11.0}/docs/pfizer-protein.md

@@ -45,6 +45,11 @@ lookup does not establish all group members. Opting into unresolved mapped
 views keeps the original group measurement visible, without assigning its value
 to an individual protein. No RNA pairings are inferred in this release.
 
+The [complete search references](search-references.md) are available through
+`registry.download_reference(name)` and `registry.open_reference(name)` without
+rebuilding abundance. This includes proteins without reported representatives;
+search membership is separate from detection or protein-group assignment.
+
 Source `NA` values remain missing and numeric zeros remain measured zeros.
 Absent group/channel combinations have no observation. For dog, 64,039 combinations
 are absent and eight reported rows contain `NA`, reproducing the existing

orfdata-0.11.0/docs/pfizer-rna.md

@@ -0,0 +1,86 @@
+# Dog and mouse RNA abundance
+
+The [Oziolor et al. atlas](https://doi.org/10.1038/s41597-026-07512-9) deposits
+native gene counts alongside its [protein measurements](pfizer-protein.md).
+ORFdata preserves each species separately, using the same measurement API.
+
+| Dataset | RNA libraries | Measured genes | Observations | Reference |
+| --- | ---: | ---: | ---: | --- |
+| `pfizer-atlas-dog-rna` | 434 | 30,941 | 13,428,394 | ROS_Cfam_1.0, Ensembl 110 |
+| `pfizer-atlas-mouse-rna` | 606 | 56,051 | 33,966,906 | GRCm39, Ensembl 110 |
+
+The dog download is about 534 MB and installs to 3.41 GB; the mouse download is
+1.27 GB and installs to 8.66 GB. Queries select observations from the installed
+SQLite database without loading the whole matrix into memory.
+
+The original `rcnt` values are integer gene counts after the authors' modified
+nf-core/Salmon pipeline. They are neither TPM nor calibrated RNA copies per cell.
+Zeros stay measured zeros: 4,218,202 dog observations and 18,019,555 mouse
+observations. Every value retains the native numeric text and source line.
+
+```sh
+pip install 'orfdata[data]'
+orfdata datasets download pfizer-atlas-dog-rna --version 2026-10-12.1
+orfdata datasets query pfizer-atlas-dog-rna --sample 1 --unit counts
+```
+
+```python
+from orfdata import DatasetRegistry, ScopedTarget
+
+registry = DatasetRegistry()
+dataset = registry.open("pfizer-atlas-dog-rna", "2026-10-12.1")
+reference, = dataset.manifest.references
+gene = ScopedTarget(reference, "gene", "ENSCAFG00845000002", "ensembl")
+for observation in dataset.measurements(target=gene, unit="counts"):
+    print(observation.sample_id, observation.value, observation.status)
+```
+
+All source gene IDs were checked against the checksum-pinned Ensembl 110 GTF.
+The import retains the source's coding and noncoding genes without a
+protein-coding filter.
+With that annotation already installed in PyEnsembl, construct the same target
+from a gene object:
+
+```python
+from pyensembl import EnsemblRelease
+
+genome = EnsemblRelease(110, species="dog")
+feature = genome.gene_by_id("ENSCAFG00845000002")
+gene = ScopedTarget.from_pyensembl(feature, kind="gene", namespace="ensembl",
+                                 context=reference)
+```
+
+An incompatible species, assembly or annotation release fails explicitly.
+There are 180 dog and 890 mouse annotation genes outside the measured count
+universe; querying one yields no observation, not a synthesized zero. Gene RNA
+abundance is context for a parent RNA and does not establish the abundance or
+translation of any particular uORF, ncORF or transcript isoform.
+
+Samples retain original numeric SIDs, GEO/BioSample accessions, SRA runs, batches,
+native tissue labels and the public source metadata. Mouse libraries include
+308 source-labelled `c57` and 298 `cd1` samples; these are biological strain
+labels, independent of the annotation reference. Donor IDs and sex are unknown
+at the deposited sample level. More specific GEO tissue labels remain alongside
+the count table's broader labels. There is no verified RNA-library-to-protein-pool
+crosswalk, so this release creates no assay pairs or protein-per-RNA ratios.
+
+GSE325522, GSE325523 and GSE325525 each carry the same decompressed count matrix.
+ORFdata pins one input and records the equivalent mirrors instead of counting
+the data three times. The full matrix contains 2,183 nonhuman libraries. Although
+the metadata lists 44 human libraries, neither the native CSV nor the deposited
+human-ortholog RDS contains their counts. That [source gap](https://github.com/openvax/orfdata/issues/37)
+remains explicit. Rat, pig and macaque rows are present in the original matrix
+and remain candidates for separate releases through the same importer.
+
+`registry.build(name, source_paths=...)` or `orfdata datasets build NAME` rebuilds
+from pinned counts, GEO metadata, factual Table S5 sample records and the
+historical GTF; workbook rebuilding needs `orfdata[build-data]`. Python returns
+a `QuantitativeDataset` ready to query. The CLI returns its path; explicit CLI
+queries of local builds are tracked in [#43](https://github.com/openvax/orfdata/issues/43).
+Normal registry queries open the independently verified published artifact.
+
+GEO data and Ensembl annotations retain their source reuse terms and attribution.
+The paper supplement has separate CC-BY-NC-ND-4.0 terms; the original workbook
+is not bundled in the quantitative artifact. Source hashes, terms and processing
+details are in each manifest. All acquisition is explicit; installed queries
+and metadata discovery work offline.

orfdata-0.11.0/docs/search-references.md

@@ -0,0 +1,75 @@
+# Native protein search references
+
+A search reference lists the sequences available to a particular MS search.
+It is independent of reported abundance or experimental evidence that a protein
+exists. ORFdata exposes the exact deposited FASTA, including unreported entries.
+
+| Dataset | Search entries | FASTA bytes, downloaded and installed |
+| --- | ---: | ---: |
+| `pfizer-atlas-human-protein` | 20,600 | 13,831,073 |
+| `pfizer-atlas-mouse-protein` | 21,989 | 14,325,720 |
+| `pfizer-atlas-dog-protein` | 20,649 | 14,659,832 |
+
+These are uncompressed, no-isoform UniProt search sets from data release
+`2026-10-12.1`. The historical UniProt release and genomic assembly remain
+unknown. ORFdata does not replace them with a current proteome.
+
+```python
+from orfdata import DatasetRegistry, ScopedTarget
+
+registry = DatasetRegistry()
+name, version = "pfizer-atlas-dog-protein", "2026-10-12.1"
+spec, = registry.search_references(name, version)  # Metadata only; no I/O.
+registry.download_reference(name, version)        # Explicit verified download.
+search = registry.open_reference(name, version)   # Offline; raises if missing.
+
+entry = search.get("A0A068LEV2")
+print(entry.header, entry.sequence, entry.sequence_sha256)
+assert search.membership(entry.target).status == "present"
+for entry in search.entries():
+    assert entry.target.reference == search.reference
+
+absent = ScopedTarget(search.reference, "protein", "not-in-this-search", "uniprot")
+assert search.membership(absent).status == "not_in_search"
+```
+
+Dog has 9,929 reported group representatives and 10,720 other search entries,
+including `A0A068LEV2`. Those other entries have **no reported representative**;
+that does not prove that their peptides were undetected. The summary table does
+not resolve all protein-group members. Query group abundance separately through
+[`dataset.measurements()`](pfizer-protein.md).
+
+Membership accepts an explicitly scoped native protein target. A different
+reference raises an error. Other namespaces, ORFs, genes and protein groups
+return `unknown`: first supply an explicit, validated reference mapping. Search
+membership never creates a zero, allocates a group's abundance, establishes
+protein detectability, or supplies MHC presentation evidence. Exact accession
+lookup does not perform sequence matching or infer coverage of a new ORF.
+
+Each entry retains its full header, FASTA ID, residues, sequence hash, source
+taxon, optional gene label and source line. `search.spec.to_dict()` records the
+dataset version, reference identity, sequence-set SHA-256, source URL, size,
+licence and attribution. The deposited records do not specify decoy/control
+roles; those fields stay `None`. UniProt records retain CC-BY-4.0 attribution.
+
+```sh
+orfdata datasets references pfizer-atlas-dog-protein --version 2026-10-12.1
+orfdata datasets download-reference pfizer-atlas-dog-protein --version 2026-10-12.1
+orfdata datasets inspect-reference pfizer-atlas-dog-protein --version 2026-10-12.1
+orfdata datasets query-reference pfizer-atlas-dog-protein --version 2026-10-12.1 --target A0A068LEV2
+```
+
+Pin the quantitative dataset version and retain the matching search spec when
+exporting an analysis. Both use the same native `ReferenceContext`; select
+`reference=spec.reference` and `asset_id=spec.source.asset_id` if a dataset has
+multiple declared search sets (`--reference KEY --asset ID` in the CLI).
+Reference downloads use a separate DataCache store and never rebuild or download
+abundance. Local source overrides are checksum-verified; `force=True` creates a
+new immutable bundle and preserves already-open readers. Ordinary discovery,
+inspection and opening stay offline. Published quantitative hashes are unchanged.
+
+The same reference model accepts any taxon, with no inferred cross-species joins.
+Adapters must explicitly declare their native search sets and formats. The
+initial parser supports deposited UniProt FASTAs. CoPY yeast's QconCAT FASTA is
+synthetic calibration material, so it is not exposed as a native yeast proteome;
+that historical search reference remains an explicit source gap.

{orfdata-0.9.0 → orfdata-0.11.0}/orfdata/__init__.py

@@ -35,6 +35,7 @@ from .pairing import (
 )
 from .reference import ReferenceContext, ScopedTarget
 from .reference_mapping import MappedMeasurement, ReferenceMapping
+from .search_reference import SearchMembership, SearchReference, SearchReferenceSpec, SearchSequence
 from .version import __version__
 
 __all__ = [
@@ -66,6 +67,10 @@ __all__ = [
     "compare_orthologs",
     "ReferenceMapping",
     "MappedMeasurement",
+    "SearchReferenceSpec",
+    "SearchReference",
+    "SearchSequence",
+    "SearchMembership",
     "load_catalog",
     "UORFCatalog",
     "UORFRecord",