orfdata 0.42.3__tar.gz → 0.42.4__tar.gz

{orfdata-0.42.3/orfdata.egg-info → orfdata-0.42.4}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: orfdata
-Version: 0.42.3
+Version: 0.42.4
 Summary: Reference proteomes with evidence-backed noncanonical ORFs on PyEnsembl genomes
 Author: OpenVax
 License-Expression: Apache-2.0

{orfdata-0.42.3 → orfdata-0.42.4}/docs/catalogs.md

@@ -82,29 +82,19 @@ pip install "orfdata"
 orfdata setup --species mouse --accept-terms   # Three catalogues and the chains
 ```
 
-`orfdata setup` downloads the published Chang et al. catalogue, builds the other
-registered mouse catalogues from their pinned sources, and downloads the chain files (`--accept-terms` for UCSC's non-commercial
+`orfdata setup` downloads the published Chang et al. and RibORF catalogues, builds OpenProt from their pinned sources, and downloads the chain files (`--accept-terms` for UCSC's non-commercial
 GRCm38/GRCm39 chains). It prints what it did with each: built, downloaded,
 already installed, repaired or skipped. If one fails, the rest are still
 installed and it exits with status 1, naming each failure; in Python,
 `install_species` raises `InstallError`, whose `result` holds the same
-report. `load_catalog(species="mouse")` then merges the three,
-in this order, into 60,707 ORFs: 2,976 MS-supported OpenProt ORFs, 16,484
-Ribo-seq ORFs of Chang et al. and 51,991 of Yang et al. 10,448 of Chang's and
-Yang's ORFs are the same ORF (28 of them also in OpenProt), and 296 OpenProt
-ORFs have Ribo-seq support from one or both, 242 from Yang et al. alone. On
-Ensembl 115 (GRCm39):
-
-| Evidence | Selected | Added to the proteome |
-| --- | ---: | ---: |
-| `"ms"` (default) | 2,976 | 2,854 |
-| `RiboSeq()` | 58,109 | 57,992 |
-| `RiboSeq() & MS()` | 378 | 371 |
-| `None` | 60,707 | 60,475 |
-
-Loading the merged catalogue takes about 10 seconds and 1.1 GB of memory, once
-per process, whichever evidence is selected. On GRCm38 and NCBIM37 all three
-are [lifted automatically](#every-assembly-of-a-species).
+report. `load_catalog(species="mouse")` merges OpenProt, Chang et al. and
+Yang et al. in that order. Current inputs contain 2,976 MS-supported OpenProt
+ORFs, 16,484 Chang et al. ORFs and 51,831 validated Yang et al. target placements.
+Overlapping source models merge by coding path and protein while keeping their
+original evidence. Merged counts and target-genome placement counts depend on
+these versions; inspect the resulting catalogue and proteome layer receipts.
+On GRCm38 and NCBIM37 the catalogues are
+[lifted automatically](#every-assembly-of-a-species).
 
 ### Mouse Ribo-seq ORFs (Chang et al.)
 
@@ -143,48 +133,55 @@ ATG, CTG, GTG or TTG. Its types are uORF (8,294), lncORF (3,576), uoORF
 
 ### Mouse RibORF ORFs (Yang et al.)
 
-`riborf-mouse` holds the mouse ORFs that RibORF 2.0 identified as translated in
-the Ribo-seq data of Yang et al. (Nat Commun 2024,
-[doi:10.1038/s41467-024-46240-9](https://doi.org/10.1038/s41467-024-46240-9);
-224 samples in 26 GEO series): uORF (26,052), lncORF (8,149), iORF (6,155),
-dORF (5,174), uoORF (3,157), N-terminal truncation (2,059), doORF (670) and
-extension (575). The source's ouORF, odORF and
-noncoding types take Chang et al.'s names, so `--biotype uoORF` finds both
-sources' overlapping uORFs. Annotated CDSs, likely sequencing errors,
-and stop-codon readthrough are not imported. Release 2 retains 11,902 valid
-ORFs shorter than five residues. Release 1, which applied a five-residue import
-floor, remains available explicitly as `--version 1`.
-
-- **Placement.** The source gives mm10 / GENCODE vM25 transcript coordinates
-  (a 1-based start, an exclusive end) and no proteins. Rather than lift mm10
-  through a restricted chain, each ORF is placed on the same transcript in
-  Ensembl 107 GRCm39 and translated from that transcript's sequence. Of the
-  54,128 non-skipped source rows, 51,991 resolve. Listed as unresolved, with the
-  reason: 1,642 rows on transcripts that changed length since GENCODE
-  vM25, 399 on transcripts that Ensembl 107 has in its GTF but in
-  neither FASTA, 95 on transcripts that are gone, and one whose
-  transcript's sequence changed at the same length. Placing through the
-  transcript gives the same coding path as lifting the source's mm10 blocks
-  through UCSC's chain for all but 2, in a duplicated region of chr14 where
-  the chain is ambiguous. Of the 12,222 source rows below five residues,
-  11,902 resolve and 320 carry one of the same explicit failures. In all,
-  51,881 place on Ensembl 115.
-- **Sources.** Supplementary Data 3 (CC BY 4.0) and Ensembl 107's GTF, cDNA
-  and ncRNA FASTAs; about 115 MB, of which the 32 MB GTF is the one Chang et
-  al.'s catalogue uses, so `orfdata setup` downloads it once.
-- **Evidence.** Presence in the source's set is aggregate Ribo-seq support:
-  `RiboSeq()` selects every ORF, `RiboSeq(min_studies=2)` none.
-- **Length policy.** `catalog.query(min_protein_length=5)` and
-  `Proteome.build(..., min_protein_length=5)` reproduce a downstream
-  five-residue selection without removing short models from the catalogue.
-  Proteome layers record the threshold and number filtered.
-- **IDs.** `riborf_` and the GRCm39 start and stop, such as
-  `riborf_4_120262600_120262620_+`, never a name Chang et al. use. Two paths
-  with the same start and stop add their transcript (`…@ENSMUST00000226495.2`),
-  and a lookup by the shared name lists both, as for OpenProt's split
-  accessions. An ORF in both sets merges by its coding path and protein,
-  whatever its name, and keeps Chang et al.'s ID; its `riborf_` ID still
-  finds it.
+`riborf-mouse` generation 3 reconstructs the ORFs reported by Yang et al.
+(Nat Commun 2024,
+[doi:10.1038/s41467-024-46240-9](https://doi.org/10.1038/s41467-024-46240-9))
+against their original GENCODE M25/GRCm38 reference before any placement on
+Ensembl 107/GRCm39. The original table supplies RNA intervals and BED blocks,
+not protein sequences. All 54,128 non-skipped rows reconstruct against the
+pinned native RNA, with the full transcript version, length, start/stop and
+every BED block checked.
+
+- **Download.** `orfdata data fetch riborf-mouse` installs the verified 24.9 MB
+  catalogue. Rebuilding remains available from its pinned source files.
+- **Target placement.** Exact full RNA, coding nucleotide sequence and translated
+  protein must agree. All target cDNA and ncRNA sequences are searched, including
+  duplicate loci and renamed transcripts. A surviving stable ID, unchanged RNA
+  length or matching start/stop alone cannot transfer evidence.
+- **Coverage.** 51,831 rows have a unique validated target placement, representing
+  41,882 distinct proteins. Another 118 have multiple exact target placements;
+  2,179 lack an unchanged complete target RNA. These 2,297 retain their native
+  models, sequences, evidence and explicit mapping reasons in
+  `catalog.metadata["unresolved"]`. `extra_orfs(...).unlifted` also reports them
+  under the requested evidence rule. A changed RNA with an unchanged small CDS
+  remains unresolved under this conservative placement method.
+- **Provenance.** Placed records retain their native `ORFRecord` in
+  `record.lifted_from` and a typed `record.reference_mapping`. The latter records
+  source/target references, original row ID, RNA interval, native CDS, source
+  pins, coordinate checks and exact sequence comparisons. Evidence summaries
+  continue to name the native reference and protein. No chain file is invented;
+  `record.liftover` is `None`. JSON and TSV exports retain both objects.
+  Managed manifests keep target annotation contexts in `references` and native
+  evidence contexts in `source_references`.
+- **Sources.** Supplementary Data 3 (CC BY 4.0), native GENCODE M25 GTF/RNA and
+  Ensembl 107 GTF/cDNA/ncRNA: about 198 MB in total. The 32 MB target GTF is also
+  used by Chang et al., so species setup acquires it once.
+- **Evidence and length.** The 224 samples in 26 GEO series provide aggregate
+  RibORF presence, not per-ORF named studies. `RiboSeq()` selects the target
+  records; `MS()` and `RiboSeq(min_studies=2)` select none. The catalogue retains
+  11,858 target ORFs shorter than five residues. Consumers choose length limits
+  when querying or building proteomes.
+- **IDs and earlier generations.** IDs are `riborf_` followed by the original
+  source row ID, including its native transcript version and RNA interval.
+  They do not relabel a current coordinate as an original observation.
+  Generations 1/2 reconstructed proteins directly from the target annotation;
+  new builds of those unvalidated generations fail before downloads and direct
+  callers to generation 3. Archived files remain historical data, with their
+  saved provenance; they are not retroactively sequence-validated.
+
+The source's ouORF, odORF and noncoding categories use the common uoORF, doORF
+and lncORF names. Canonical CDSs, likely sequencing errors and stop-codon
+readthrough rows remain excluded.
 
 ## Mouse ORFs from OpenProt
 

{orfdata-0.42.3 → orfdata-0.42.4}/orfdata/catalog.py

@@ -11,7 +11,7 @@ import os
 import re
 from collections import Counter
 from copy import copy, deepcopy
-from dataclasses import asdict, dataclass, replace
+from dataclasses import asdict, dataclass, field, replace
 from importlib.resources import files
 from pathlib import Path
 from typing import NamedTuple, Optional
@@ -22,8 +22,10 @@ from .evidence import MS_METHODS, evidence_rule, stored_method
 from .evidence_summary import EvidenceSummary, unique_summaries
 from .initiation_context import KOZAK_PREFERENCES, InitiationContext
 from .liftover import Liftover
+from .measurements import _plain
 from .model import ORFModel, TranscriptORF, _contig, _id_forms, _species, _transcript_match
-from .reference import ReferenceContext
+from .reference import ReferenceContext, ScopedTarget
+from .reference_mapping import ReferenceMapping
 from .sequence import genetic_code, reverse_complement_dna
 
 
@@ -142,17 +144,35 @@ class ORFRecord:
     members: tuple = ()
     lifted_from: Optional["ORFRecord"] = None
     liftover: Optional[Liftover] = None
+    reference_mapping: Optional[ReferenceMapping] = field(default=None, hash=False)
 
     def __post_init__(self):
         object.__setattr__(self, "summaries", unique_summaries(self.summaries))
         object.__setattr__(self, "members", tuple(self.members))
-        if (self.lifted_from is None) != (self.liftover is None):
-            raise ValueError("A lifted record names both its original record and the chain")
         original = self.lifted_from
+        methods = int(self.liftover is not None) + int(self.reference_mapping is not None)
+        if methods != int(original is not None):
+            raise ValueError("A placed record names its original and exactly one chain or reference mapping")
         if original is not None and any(getattr(self, name) != getattr(original, name) for name in (
                 "orf_id", "protein_sequence", "translation_table", "start_codon", "source",
                 "ribosome_studies", "peptides", "summaries")):
             raise ValueError(f"A lifted record keeps its original's ID, protein and evidence: {self.orf_id}")
+        mapping = self.reference_mapping
+        if mapping is not None:
+            if not isinstance(mapping, ReferenceMapping):
+                raise TypeError("ORF reference placements require a ReferenceMapping")
+            if original.reference_context is None or self.reference_context is None:
+                raise ValueError("ORF reference placements require both reference identities")
+            source = ScopedTarget(original.reference_context, "orf", original.orf_id, original.source)
+            target = ScopedTarget(self.reference_context, "orf", self.orf_id, self.source)
+            if mapping.source != source or mapping.targets != (target,) or mapping.status != "unique":
+                raise ValueError("ORF reference mapping endpoints disagree with the records")
+            digest = hashlib.sha256(self.protein_sequence.encode("ascii")).hexdigest()
+            if not mapping.sequence_validated or mapping.validation["sequence"]["source_sha256"] != digest:
+                raise ValueError("ORF reference mapping protein identity disagrees with the records")
+            expected = {"source": _placement_position(original), "target": _placement_position(self)}
+            if _plain(mapping.validation.get("position")) != expected:
+                raise ValueError("ORF reference mapping positions disagree with the records")
         if self.members:
             # Each member validated its own summaries against its own identity.
             key = _merge_key(self)
@@ -229,7 +249,7 @@ class ORFRecord:
 
     def to_dict(self):
         """Return a serializable copy of this record."""
-        result = asdict(self)
+        result = _plain(self)
         if not self.summaries:
             result.pop("summaries")
         if self.reference_context is None:
@@ -246,7 +266,10 @@ class ORFRecord:
             result["lifted_from"] = self.lifted_from.to_dict()
         else:
             result.pop("lifted_from")
+        if self.liftover is None:
             result.pop("liftover")
+        if self.reference_mapping is None:
+            result.pop("reference_mapping")
         return result
 
     def map_initiation_position(self, position, *, genome_build):
@@ -417,6 +440,12 @@ class ORFRecord:
         }
 
 
+def _placement_position(record):
+    """The coding placement bound by a typed source-to-target reference mapping."""
+    return dict(transcript_id=record.transcript_id, contig=record.contig, strand=record.strand,
+                coding_blocks=[list(block) for block in record.coding_blocks])
+
+
 def _qualify_colliding_ids(records):
     """Give colliding source records unique catalogue IDs, retaining their native records."""
     records = tuple(records)
@@ -436,7 +465,7 @@ def _qualify_colliding_ids(records):
         names = [f"{quote(r.source, safe='')}:{quote(r.orf_id, safe='')}" if i in qualified
                  else r.orf_id for i, r in enumerate(records)]
     return tuple(r if name == r.orf_id else replace(
-        r, orf_id=name, members=r.members or (r,), lifted_from=None, liftover=None)
+        r, orf_id=name, members=r.members or (r,), lifted_from=None, liftover=None, reference_mapping=None)
         for r, name in zip(records, names))
 
 
@@ -449,6 +478,15 @@ class ORFCatalog:
         self.genome_build = metadata["genome_build"]
         self.taxon_id = metadata["taxon_id"]
         self.species = metadata["species"]
+        for entry in metadata.get("unresolved", ()):
+            if entry.get("native_record") is not None:
+                native = _record(entry["native_record"])
+                mapping = ReferenceMapping.from_dict(entry["reference_mapping"])
+                expected = ScopedTarget(native.reference_context, "orf", native.orf_id, native.source)
+                if (mapping.source != expected or mapping.status == "unique"
+                        or mapping.target_reference.taxon_id != self.taxon_id
+                        or mapping.target_reference.genome_build != self.genome_build):
+                    raise ValueError("Unresolved reference mapping disagrees with its native record or catalogue")
         self._by_id = {r.orf_id: r for r in self.records}
         self._aliases = {}
         self._by_source_id = {}
@@ -834,7 +872,7 @@ def merge_catalogs(catalogs):
         pinned = {None if m.reference_context is None else m.reference_context.key for m in members}
         records.append(replace(
             # The merged ORF is no one source's lift; each member keeps its own lift provenance.
-            primary, members=tuple(members), lifted_from=None, liftover=None,
+            primary, members=tuple(members), lifted_from=None, liftover=None, reference_mapping=None,
             initiation_context=_shared_context(primary, members),
             # Sources pinned to different annotations of one assembly leave the ORF unpinned.
             reference_context=primary.reference_context if len(pinned) == 1 else None, **_union(members, study)))
@@ -927,7 +965,10 @@ def _record(row, legacy=False):
     row["members"] = tuple(_record(m) for m in row.get("members", ()))
     if row.get("lifted_from") is not None:
         row["lifted_from"] = _record(row["lifted_from"])
+    if row.get("liftover") is not None:
         row["liftover"] = Liftover(**row["liftover"])
+    if row.get("reference_mapping") is not None:
+        row["reference_mapping"] = ReferenceMapping.from_dict(row["reference_mapping"])
     return ORFRecord(**row)
 
 

{orfdata-0.42.3 → orfdata-0.42.4}/orfdata/catalog_manifest.py

@@ -21,14 +21,22 @@ class CatalogManifest(_Record):
     coverage: Mapping
     processing: Mapping
     schema: str = "orfdata.catalog-manifest.v1"
+    source_references: tuple = ()
 
     @classmethod
     def from_dict(cls, data):
         values = dict(data)
         values["references"] = tuple(ReferenceContext(**c) for c in values["references"])
         values["sources"] = tuple(SourceAsset.from_dict(s) for s in values["sources"])
+        values["source_references"] = tuple(ReferenceContext(**c) for c in values.get("source_references", ()))
         return cls(**values)
 
+    def to_dict(self):
+        result = super().to_dict()
+        if not self.source_references:
+            result.pop("source_references")
+        return result
+
     def __post_init__(self):
         for name in ("catalog_id", "version", "title"):
             _identifier(getattr(self, name), name)
@@ -54,6 +62,14 @@ class CatalogManifest(_Record):
         builds = {r.genome_build for r in self.references if r.genome_build is not None}
         if len(builds) != 1 or not any(r.is_genomic for r in self.references):
             raise ValueError("A catalogue must have one assembly and a complete genomic reference")
+        if isinstance(self.source_references, str):
+            raise TypeError("Source references must be a sequence of contexts")
+        object.__setattr__(self, "source_references", tuple(self.source_references))
+        if any(not isinstance(r, ReferenceContext) or not r.is_genomic
+               or r.taxon_id != self.taxon_id for r in self.source_references):
+            raise ValueError("Native source references must be genomic contexts of the catalogue species")
+        if len({r.key for r in self.source_references}) != len(self.source_references):
+            raise ValueError("Duplicate native source reference contexts")
         if any(not isinstance(s, SourceAsset) for s in self.sources):
             raise TypeError("Catalogue sources must be SourceAssets")
         if len({s.asset_id for s in self.sources}) != len(self.sources):
@@ -91,9 +107,16 @@ class CatalogManifest(_Record):
             reference = record.reference_context
             if reference is None or not reference.is_genomic or reference.key not in references:
                 raise ValueError("Managed ORF records require a declared complete genomic reference")
+        evidence_references = references | {r.key for r in self.source_references}
         for summary in catalog.evidence_summaries:
-            if summary.target.reference.key not in references:
+            if summary.target.reference.key not in evidence_references:
                 raise ValueError("Evidence summary reference is absent from catalogue manifest")
+        for entry in catalog.metadata.get("unresolved", ()):
+            if entry.get("native_record") is not None:
+                native = ReferenceContext(**entry["native_record"]["reference_context"])
+                target = ReferenceContext(**entry["reference_mapping"]["target_reference"])
+                if native.key not in evidence_references or target.key not in references:
+                    raise ValueError("Unresolved placement reference is absent from catalogue manifest")
         counts = dict(distinct_protein_sequences=len({r.protein_sequence for r in catalog.records}),
                       evidence_summaries=len(catalog.evidence_summaries))
         for key, actual in counts.items():

{orfdata-0.42.3 → orfdata-0.42.4}/orfdata/catalogs.py

@@ -40,7 +40,7 @@ def _copy_catalog(inputs, output, manifest):
         handle.write(json.dumps(catalog.to_dict(), sort_keys=True, separators=(",", ":"), allow_nan=False).encode())
 
 
-def _builder(manifest):
+def _builder(manifest, *, for_build=False):
     """The packaged importer a release names; import paths in metadata are never executed."""
     importer = manifest.processing.get("importer")
     if importer == "catalog-copy-v1":
@@ -54,7 +54,13 @@ def _builder(manifest):
 
         return lambda inputs, output: import_orfs(inputs, output, manifest)
     if importer in ("riborf-v1", "riborf-v2"):
-        from .sources.riborf import import_orfs
+        from .sources.riborf import import_orfs, validate_transcript_manifest
+
+        if for_build:
+            validate_transcript_manifest(manifest)
+        return lambda inputs, output: import_orfs(inputs, output, manifest)
+    if importer == "riborf-native-v3":
+        from .sources.riborf_native import import_orfs
 
         return lambda inputs, output: import_orfs(inputs, output, manifest)
     if importer in ("riborf-genome-v1", "riborf-genome-v2"):
@@ -198,7 +204,7 @@ class CatalogRegistry:
     def build(self, name, version=None, *, force=False, source_paths=None, download_options=None):
         """Explicitly materialize an allowlisted catalogue source adapter."""
         manifest = self.info(name, version)
-        return build_catalog(manifest, _builder(manifest), cache_root=self._cache_root,
+        return build_catalog(manifest, _builder(manifest, for_build=True), cache_root=self._cache_root,
                              source_paths=source_paths, force=force, download_options=download_options)
 
     def inspect_build(self, name, version=None, *, verify_files=True):

{orfdata-0.42.3 → orfdata-0.42.4}/orfdata/cli.py

@@ -320,14 +320,14 @@ def _query(args, parser, catalog_loader):
             "initiation_context_json",
         )
         scoped = not isinstance(catalog, UORFCatalog)
-        provenance = ("members", "liftover", "lifted_from")
+        provenance = ("members", "liftover", "lifted_from", "reference_mapping")
         if scoped:
             fields += ("start_codon", "reference_context_json", "summaries_json") + tuple(
                 f + "_json" for f in provenance)
         elif any(r.summaries for r in records):
             # The released upstream-only columns have no place for aggregate counts.
             parser.error("The legacy upstream-ORF TSV cannot export aggregate evidence; use --format json")
-        elif any(r.members or r.liftover for r in records):
+        elif any(r.members or r.lifted_from for r in records):
             parser.error("The legacy upstream-ORF TSV cannot export merged or lifted records; use --format json")
         elif any(getattr(r, "start_codon", None) for r in records):
             parser.error("The legacy upstream-ORF TSV cannot export declared start codons; use --format json")

{orfdata-0.42.3 → orfdata-0.42.4}/orfdata/data/catalogs.json

@@ -205,7 +205,7 @@
     "priority": 20
   },
   "riborf-mouse": {
-    "default_version": "2",
+    "default_version": "3",
     "priority": 30,
     "versions": {
       "1": {
@@ -408,6 +408,151 @@
           "redistributable": true
         },
         "assets": {}
+      },
+      "3": {
+        "manifest": {
+          "catalog_id": "riborf-mouse",
+          "version": "3",
+          "title": "Mouse RibORF 2.0 translated ORFs of Yang et al. 2024",
+          "references": [
+            {
+              "taxon_id": 10090,
+              "species": "mus_musculus",
+              "genome_build": "GRCm39",
+              "annotation_name": "ensembl",
+              "annotation_version": "107",
+              "assembly_accession": null,
+              "strain": null,
+              "strain_taxon_id": null,
+              "sequence_sha256": null,
+              "taxonomy_version": "ncbi-2026-10-12",
+              "source_sha256": null
+            }
+          ],
+          "sources": [
+            {
+              "asset_id": "table",
+              "url": "https://static-content.springer.com/esm/art%3A10.1038%2Fs41467-024-46240-9/MediaObjects/41467_2024_46240_MOESM6_ESM.xlsx",
+              "sha256": "4dc8444c2fd3da25c7066074c71d1ec5427afd5d005862c6c61dc3f27fb9d32c",
+              "size": 24211005,
+              "access_terms": "Supplementary Data 3 of Yang et al., Nat Commun 2024, CC BY 4.0.",
+              "license_url": null,
+              "redistributable": true,
+              "metadata": {}
+            },
+            {
+              "asset_id": "gtf",
+              "url": "https://ftp.ensembl.org/pub/release-107/gtf/mus_musculus/Mus_musculus.GRCm39.107.gtf.gz",
+              "sha256": "15af16b7bef88c4309a82d49579fdd967f6ac8769b7f0bf657bbf7ee73b779e3",
+              "size": 31861821,
+              "access_terms": "Ensembl imposes no restrictions on access to, or use of, the data provided.",
+              "license_url": null,
+              "redistributable": true,
+              "metadata": {}
+            },
+            {
+              "asset_id": "cdna",
+              "url": "https://ftp.ensembl.org/pub/release-107/fasta/mus_musculus/cdna/Mus_musculus.GRCm39.cdna.all.fa.gz",
+              "sha256": "34499a727ff72bdddb2f32e5831b14fb5a9d20218cd79b8639dc8736b2c596c0",
+              "size": 51269494,
+              "access_terms": "Ensembl imposes no restrictions on access to, or use of, the data provided.",
+              "license_url": null,
+              "redistributable": true,
+              "metadata": {}
+            },
+            {
+              "asset_id": "ncrna",
+              "url": "https://ftp.ensembl.org/pub/release-107/fasta/mus_musculus/ncrna/Mus_musculus.GRCm39.ncrna.fa.gz",
+              "sha256": "0472b7d912e84d838485ecf236670a68319a32fb4580062cd9768c7bce85d9ab",
+              "size": 7668482,
+              "access_terms": "Ensembl imposes no restrictions on access to, or use of, the data provided.",
+              "license_url": null,
+              "redistributable": true,
+              "metadata": {}
+            },
+            {
+              "asset_id": "source_gtf",
+              "url": "https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M25/gencode.vM25.annotation.gtf.gz",
+              "sha256": "e6c9704cb5832b8a1c71431ecac9ea30057377c72cdc9a12dc5d0405ab066d5a",
+              "size": 28542432,
+              "access_terms": "GENCODE reference annotation/transcript data under EMBL-EBI data-use terms.",
+              "license_url": "https://www.ebi.ac.uk/about/terms-of-use/",
+              "redistributable": true,
+              "metadata": {}
+            },
+            {
+              "asset_id": "source_rna",
+              "url": "https://ftp.ebi.ac.uk/pub/databases/gencode/Gencode_mouse/release_M25/gencode.vM25.transcripts.fa.gz",
+              "sha256": "5139cf409a680b60434716ca10ad4e474cbbe5d2e4d11af7553edfad8ed73670",
+              "size": 54587418,
+              "access_terms": "GENCODE reference annotation/transcript data under EMBL-EBI data-use terms.",
+              "license_url": "https://www.ebi.ac.uk/about/terms-of-use/",
+              "redistributable": true,
+              "metadata": {}
+            }
+          ],
+          "citations": [
+            "Yang H, Li Q, Stroup EK, Wang S, Ji Z. Widespread stable noncanonical peptides identified by integrated analyses of ribosome profiling and ORF features. Nat Commun. 2024;15:1932. doi:10.1038/s41467-024-46240-9"
+          ],
+          "coverage": {
+            "records": 51831,
+            "distinct_protein_sequences": 41882,
+            "evidence_summaries": 51831,
+            "source_rows": 54128,
+            "short_rows": 12222,
+            "native_valid_rows": 54128,
+            "resolved_rows": 51831,
+            "unresolved_rows": 2297,
+            "short_records": 11858
+          },
+          "processing": {
+            "importer": "riborf-native-v3",
+            "sheet": "mouse",
+            "source": "riborf-mouse",
+            "release": "NatCommun-15-1932",
+            "skip_types": [
+              "canonical",
+              "seqerror",
+              "readthrough"
+            ],
+            "biotypes": {
+              "ouORF": "uoORF",
+              "odORF": "doORF",
+              "noncoding": "lncORF"
+            },
+            "scope": "Native GENCODE vM25/mm10 ORFs reconstructed first; target Ensembl 107/GRCm39 placement requires exact full RNA, CDS and protein, with every source BED block validated. Aggregate presence only.",
+            "report_protein_lengths_below": 5,
+            "source_annotation": "GENCODE vM25"
+          },
+          "schema": "orfdata.catalog-manifest.v1",
+          "source_references": [
+            {
+              "taxon_id": 10090,
+              "species": "mouse",
+              "genome_build": "GRCm38",
+              "annotation_name": "gencode",
+              "annotation_version": "M25",
+              "assembly_accession": null,
+              "strain": null,
+              "strain_taxon_id": null,
+              "sequence_sha256": "5139cf409a680b60434716ca10ad4e474cbbe5d2e4d11af7553edfad8ed73670",
+              "taxonomy_version": "ncbi-2026-10-12",
+              "source_sha256": "e6c9704cb5832b8a1c71431ecac9ea30057377c72cdc9a12dc5d0405ab066d5a"
+            }
+          ]
+        },
+        "distribution": {
+          "access_terms": "Built from pinned Yang et al. Supplementary Data 3 (CC BY 4.0), native GENCODE vM25 RNA/GTF, and Ensembl 107 target annotation/RNA. Original references and evidence retained; exact-sequence placement only.",
+          "redistributable": true,
+          "data_release": "data-riborf-mouse-3"
+        },
+        "assets": {
+          "catalog.json.gz": {
+            "url": "https://github.com/openvax/orfdata/releases/download/data-riborf-mouse-3/riborf-mouse-3.json.gz",
+            "sha256": "2d60d7783821f3ab88485197b141c355ca07fbe6e5a9da78cec42262508c5d03",
+            "size": 24875346
+          }
+        }
       }
     }
   },

{orfdata-0.42.3 → orfdata-0.42.4}/orfdata/liftover.py

@@ -374,7 +374,8 @@ def _lift(record, chain):
                                   source_sha256=chain.liftover.sha256)
     lifted = replace(record, contig=contig, strand=strand, coding_blocks=blocks,
                      genome_build=chain.liftover.target_build, reference_context=pinned,
-                     initiation_context=None, members=members, lifted_from=record, liftover=chain.liftover)
+                     initiation_context=None, members=members, lifted_from=record, liftover=chain.liftover,
+                     reference_mapping=None)
     context = record.initiation_context
     if context is not None:
         positions = [chain.map(record.contig, p, p) for p in context.genomic_positions]
@@ -394,6 +395,9 @@ def unlifted(metadata):
     """Every record a catalogue's lifts left out, through each hop and merged input: (record, assembly, reason)."""
     from .catalog import _record
 
+    for entry in metadata.get("unresolved", ()):
+        if entry.get("native_record") is not None and entry.get("reference_mapping") is not None:
+            yield _record(entry["native_record"]), metadata["genome_build"], entry["reason"]
     lift = metadata.get("liftover")
     if lift:
         yield from unlifted(lift["input"]["metadata"])

{orfdata-0.42.3 → orfdata-0.42.4}/orfdata/proteome.py

@@ -2085,6 +2085,9 @@ def _filter_unlifted(metadata, accepts):
     """Apply a union selection to failed lifts as well as placed records."""
     from .catalog import _record
 
+    if "unresolved" in metadata:
+        metadata["unresolved"] = [entry for entry in metadata["unresolved"]
+                                  if entry.get("native_record") is None or accepts(_record(entry["native_record"]))]
     lift = metadata.get("liftover")
     if lift:
         _filter_unlifted(lift["input"]["metadata"], accepts)

{orfdata-0.42.3 → orfdata-0.42.4}/orfdata/sources/riborf.py

@@ -12,8 +12,9 @@ end, the stop codon included.
 The source gives no protein sequences, so each ORF is placed through the same
 transcript in a pinned Ensembl annotation (:func:`import_orfs`): its
 transcript coordinates map through that transcript's exons, and its protein is
-translated from that transcript's sequence. For mouse this also moves the ORFs
-from mm10 to GRCm39 without a chain file with usage restrictions. A row is
+translated from that transcript's sequence. Mouse uses the separate native
+GENCODE M25 reconstruction and exact-sequence placement in ``riborf_native``;
+the unvalidated historical mouse rebuilds are rejected here. A row is
 left out, with the reason, when the transcript is missing, has no sequence or
 changed length since the source's annotation, or when its sequence there
 doesn't start with the stated codon and end at its first stop.
@@ -110,8 +111,15 @@ def _parsed(processing, sheet_rows):
     return parsed, source_rows, short
 
 
+def validate_transcript_manifest(manifest):
+    """Reject superseded mouse builds before acquiring their source files."""
+    if manifest.references[0].taxon_id == 10090:
+        raise ValueError("Mouse RibORF generations 1/2 lack native sequence validation; build generation 3")
+
+
 def import_orfs(inputs, output, manifest):
     """Write the catalogue ``manifest`` describes from the source table and the pinned Ensembl files."""
+    validate_transcript_manifest(manifest)
     processing = manifest.processing
     reference = manifest.references[0]
     # Ensembl IDs carry a version (ENST00000000233.10); C. elegans names have dots of their own (2L52.1a.1).