orfdata 0.27.1__tar.gz → 0.28.0__tar.gz

{orfdata-0.27.1 → orfdata-0.28.0}/MANIFEST.in

@@ -8,3 +8,5 @@ recursive-include tests/data/search_reference *.gz
 recursive-include tests/data/wang *.zip *.gz
 recursive-include tests/data/ingolia *.gz *.txt
 recursive-include tests/data/openprot *.zip *.gz
+recursive-include tests/data/ncorf_mammals *.bed *.fa *.xlsx *.gz *.md
+recursive-include tests/data/riborf *.xlsx *.gz *.md

{orfdata-0.27.1/orfdata.egg-info → orfdata-0.28.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: orfdata
-Version: 0.27.1
+Version: 0.28.0
 Summary: Reference proteomes with evidence-backed noncanonical ORFs on PyEnsembl genomes
 Author: OpenVax
 License-Expression: Apache-2.0
@@ -81,16 +81,19 @@ orfdata proteome --species human --release 115 --output human-e115.fasta
 ```
 
 A translated ORF or a detected peptide does not by itself establish a stable,
-functional protein. Most MS support in the bundled catalogue comes from
-peptides presented by HLA molecules.
+functional protein. Most MS support in the bundled human catalogue comes from
+immunopeptidomics: peptides eluted from HLA molecules.
 
 ## Species
 
-The bundled noncanonical ORF catalogue is human GRCh38. Mouse ORFs come from
-OpenProt 2.2: `orfdata catalogs build openprot-mouse` builds that catalogue
-from OpenProt's own files. After `orfdata chains download` (add
+The bundled noncanonical ORF catalogue is human GRCh38. Mouse has 48,805:
+Ribo-seq ORFs of Chang et al. 2025 and Yang et al. 2024, and 2,976 MS-supported
+ones from OpenProt 2.2;
+`pip install "orfdata[data,build-data]"` then `orfdata setup --species mouse`
+builds all three from their sources' own files. After `orfdata chains download` (add
 `--accept-terms` for UCSC's non-commercial mouse chains), older assemblies
-(GRCh37, NCBI36, GRCm38, NCBIM37) use these catalogues lifted to them. Dog and yeast proteomes contain their
+(GRCh37, NCBI36, GRCm38, NCBIM37) and T2T-CHM13 use these catalogues lifted
+to them; `orfdata genomes install t2t-chm13` installs a CHM13 annotation. Dog and yeast proteomes contain their
 PyEnsembl-annotated proteins. `load_catalog(species=...)` and
 `with_extra_orfs()` pick the catalogue for a species and raise when none is
 available or built.

{orfdata-0.27.1 → orfdata-0.28.0}/README.md

@@ -51,16 +51,19 @@ orfdata proteome --species human --release 115 --output human-e115.fasta
 ```
 
 A translated ORF or a detected peptide does not by itself establish a stable,
-functional protein. Most MS support in the bundled catalogue comes from
-peptides presented by HLA molecules.
+functional protein. Most MS support in the bundled human catalogue comes from
+immunopeptidomics: peptides eluted from HLA molecules.
 
 ## Species
 
-The bundled noncanonical ORF catalogue is human GRCh38. Mouse ORFs come from
-OpenProt 2.2: `orfdata catalogs build openprot-mouse` builds that catalogue
-from OpenProt's own files. After `orfdata chains download` (add
+The bundled noncanonical ORF catalogue is human GRCh38. Mouse has 48,805:
+Ribo-seq ORFs of Chang et al. 2025 and Yang et al. 2024, and 2,976 MS-supported
+ones from OpenProt 2.2;
+`pip install "orfdata[data,build-data]"` then `orfdata setup --species mouse`
+builds all three from their sources' own files. After `orfdata chains download` (add
 `--accept-terms` for UCSC's non-commercial mouse chains), older assemblies
-(GRCh37, NCBI36, GRCm38, NCBIM37) use these catalogues lifted to them. Dog and yeast proteomes contain their
+(GRCh37, NCBI36, GRCm38, NCBIM37) and T2T-CHM13 use these catalogues lifted
+to them; `orfdata genomes install t2t-chm13` installs a CHM13 annotation. Dog and yeast proteomes contain their
 PyEnsembl-annotated proteins. `load_catalog(species=...)` and
 `with_extra_orfs()` pick the catalogue for a species and raise when none is
 available or built.

{orfdata-0.27.1 → orfdata-0.28.0}/docs/catalogs.md

@@ -7,19 +7,23 @@ the same explicit discovery, download, build and offline-open workflow as
 downloaded or built, `load_catalog(species=...)` opens it, so `extra_orfs(genome)` and
 `Proteome.with_extra_orfs()` use it without `catalog=`; pass
 `load_catalog(species=..., registry=...)` as `catalog=` for your own registry.
-A bundled catalogue is preferred, a catalogue's default release is opened
-unless only another release covers the assembly, and two registered catalogues
-for one species and assembly are never chosen between; open one by name
-instead. It returns the existing `ORFCatalog`; query results still bind to the
+Every catalogue for the species is used, merged: those on the requested
+assembly as they are, and those on another assembly the pinned chains reach,
+lifted ([details](#every-assembly-of-a-species)). Each catalogue contributes its
+release on the requested assembly if it has one, else its default release. It
+returns the existing `ORFCatalog`; query results still bind to the
 original PyEnsembl transcript through `ReferenceORF`. To use several catalogues
 for one species together, see
 [merging catalogues](proteome.md#merging-catalogues).
 
-The packaged registry has one entry, the [OpenProt mouse catalogue](#mouse-orfs-from-openprot),
-which you build from OpenProt's own files. `orfdata catalogs status` (in Python,
-`catalog_status()`) lists every catalogue `load_catalog` can open, the bundled
-one included, with its species, assembly, record count, whether it is
-installed and which one is selected for each assembly:
+The packaged registry has three mouse catalogues, each built from its sources'
+own files: [OpenProt's MS-supported ORFs](#mouse-orfs-from-openprot),
+[Chang et al.'s Ribo-seq ORFs](#mouse-ribo-seq-orfs-chang-et-al) and
+[Yang et al.'s RibORF ORFs](#mouse-riborf-orfs-yang-et-al). `orfdata setup
+--species mouse` installs all three, with the chain files. `orfdata catalogs status`
+(in Python, `catalog_status()`) lists every catalogue `load_catalog` can open,
+the bundled one included, with its species, assembly, record count, whether it
+is installed and whether it is used:
 
 ```sh
 orfdata catalogs status
@@ -61,22 +65,119 @@ evidence filters, and JSON/TSV/FASTA exports. They cannot override the selected
 release with a local `--catalog` argument. Use `orfdata --catalog FILE` for the
 existing standalone local-file workflow.
 
+## Mouse
+
+```sh
+pip install "orfdata[data,build-data]"
+orfdata setup --species mouse --accept-terms   # Three catalogues and the chains; about 130 MB of sources
+```
+
+`orfdata setup` builds each registered mouse catalogue from its pinned sources
+and downloads the chain files (`--accept-terms` for UCSC's non-commercial
+GRCm38/GRCm39 chains). It prints what it did with each: built, downloaded,
+already installed, repaired or skipped. If one fails, the rest are still
+installed and it exits with status 1, naming each failure; in Python,
+`install_species` raises `InstallError`, whose `result` holds the same
+report. `load_catalog(species="mouse")` then merges the three,
+in this order, into 48,805 ORFs: 2,976 MS-supported OpenProt ORFs, 16,484
+Ribo-seq ORFs of Chang et al. and 40,089 of Yang et al. 10,448 of Chang's and
+Yang's ORFs are the same ORF (28 of them also in OpenProt), and 296 OpenProt
+ORFs have Ribo-seq support from one or both, 242 from Yang et al. alone. On
+Ensembl 115 (GRCm39):
+
+| Evidence | Selected | Added to the proteome |
+| --- | ---: | ---: |
+| `"ms"` (default) | 2,976 | 2,854 |
+| `RiboSeq()` | 46,207 | 46,094 |
+| `RiboSeq() & MS()` | 378 | 371 |
+| `None` | 48,805 | 48,577 |
+
+Loading the merged catalogue takes about 10 seconds and 0.9 GB of memory, once
+per process, whichever evidence is selected. On GRCm38 and NCBIM37 all three
+are [lifted automatically](#every-assembly-of-a-species). Dog has no
+experimentally supported ORF catalogue from any source: OpenProt and the
+Ribo-seq resources have no dog data, and dog MS studies searched only canonical
+proteins.
+
+### Mouse Ribo-seq ORFs (Chang et al.)
+
+`ncorf-mammals-mouse` holds the mouse ncORFs of Chang et al.
+(eLife 2025, [doi:10.7554/eLife.109128](https://doi.org/10.7554/eLife.109128)):
+16,484 of the source's 16,485,
+called with PRICE, RiboCode and Ribo-TISH on Ensembl 107: an ORF is kept when at
+least two callers find it in a library, in at least two libraries, starting at
+ATG, CTG, GTG or TTG. Its types are uORF (8,294), lncORF (3,576), uoORF
+(1,950), iORF (1,749), dORF (741) and doORF (174).
+
+- **Sources.** The authors' BED and protein FASTA (MIT, pinned commit), the
+  paper's Table S3 (CC BY 4.0) and Ensembl 107's GTF; about 40 MB. Building
+  needs `orfdata[build-data]` for the spreadsheet.
+- **Coding paths.** The BED garbles multi-exon block sizes, so each ORF's
+  path comes from its representative transcript and transcript coordinates
+  in Ensembl 107, checked against its genomic start, stop, BED span and
+  protein length. 16,484 resolve; one without a stop codon is listed as
+  unresolved. All 16,484 bind to their Ensembl 107 transcript with their exact
+  coding path and protein; 16,470 place on Ensembl 115.
+- **Starts.** The source writes GTG-initiated proteins with valine. A
+  near-cognate start is read by the initiator methionine, so these proteins
+  begin with M, and their records declare the GTG start (CTG and TTG are
+  already starts of the standard code).
+- **Evidence.** Each ORF's library and caller counts become aggregate
+  Ribo-seq summaries, so `RiboSeq()` selects every ORF and `MS()` none.
+  The libraries aren't named per ORF, so `RiboSeq(min_studies=2)` selects
+  none.
+
+### Mouse RibORF ORFs (Yang et al.)
+
+`riborf-mouse` holds the mouse ORFs that RibORF 2.0 identified as translated in
+the Ribo-seq data of Yang et al. (Nat Commun 2024,
+[doi:10.1038/s41467-024-46240-9](https://doi.org/10.1038/s41467-024-46240-9);
+224 samples in 26 GEO series), of at least 5 residues: uORF (20,399), lncORF
+(6,954), dORF (4,075), uoORF (3,037), iORF (2,370), N-terminal truncation
+(2,047) and extension (575), and doORF (632). The source's ouORF, odORF and
+noncoding types take Chang et al.'s names, so `--biotype uoORF` finds both
+sources' overlapping uORFs. Annotated CDSs, likely sequencing errors,
+stop-codon readthrough and 12,222 ORFs shorter than Chang et al.'s shortest (5
+residues; 4,505 are a bare start and stop) are not imported.
+
+- **Placement.** The source gives mm10 / GENCODE vM25 transcript coordinates
+  (a 1-based start, an exclusive end) and no proteins. Rather than lift mm10
+  through a restricted chain, each ORF is placed on the same transcript in
+  Ensembl 107 GRCm39 and translated from that transcript's sequence. Of the
+  41,906 rows long enough, 40,089 resolve. Listed as unresolved, with the
+  reason: 1,393 rows on 633 transcripts that changed length since GENCODE
+  vM25, 338 on 216 TEC transcripts that Ensembl 107 has in its GTF but in
+  neither FASTA, 85 on 51 transcripts that are gone, and one whose
+  transcript's sequence changed at the same length. Placing through the
+  transcript gives the same coding path as lifting the source's mm10 blocks
+  through UCSC's chain for all but 2, in a duplicated region of chr14 where
+  the chain is ambiguous. 39,983 place on Ensembl 115.
+- **Sources.** Supplementary Data 3 (CC BY 4.0) and Ensembl 107's GTF, cDNA
+  and ncRNA FASTAs; about 115 MB, of which the 32 MB GTF is the one Chang et
+  al.'s catalogue uses, so `orfdata setup` downloads it once. Building needs
+  `orfdata[build-data]`.
+- **Evidence.** Presence in the source's set is aggregate Ribo-seq support:
+  `RiboSeq()` selects every ORF, `RiboSeq(min_studies=2)` none.
+- **IDs.** `riborf_` and the GRCm39 start and stop, such as
+  `riborf_4_120262600_120262620_+`, never a name Chang et al. use. Two paths
+  with the same start and stop add their transcript (`…@ENSMUST00000226495.2`),
+  and a lookup by the shared name lists both, as for OpenProt's split
+  accessions. An ORF in both sets merges by its coding path and protein,
+  whatever its name, and keeps Chang et al.'s ID; its `riborf_` ID still
+  finds it.
+
 ## Mouse ORFs from OpenProt
 
 `openprot-mouse` holds the alternative proteins (AltProts) and novel isoforms
 that [OpenProt 2.2](https://openprot.org) reports for mouse with at least two
 unique MS peptides, on Ensembl 106 GRCm39 transcripts. orfdata does not
 redistribute OpenProt's files, and OpenProt states no licence for them. You
-build the catalogue from OpenProt's own downloads (3.4 MB, about 5 seconds);
-each file is checked against its pinned size and SHA-256:
-
-```sh
-pip install "orfdata[data]"
-orfdata catalogs build openprot-mouse
-```
-
-After that, `load_catalog(species="mouse")`, `extra_orfs(genome)` and
-`Proteome.build(genome)` use it for any GRCm39 Ensembl release:
+build the catalogue from OpenProt's own downloads (3.4 MB, about 5 seconds),
+each checked against its pinned size and SHA-256, with `orfdata setup --species
+mouse` along with the Ribo-seq catalogue above, or alone with `orfdata catalogs
+build openprot-mouse`. `load_catalog(species="mouse")`, `extra_orfs(genome)`
+and `Proteome.build(genome)` use all three mouse catalogues, for any GRCm39
+Ensembl release, so all must be installed. With only MS-supported ORFs selected, the default:
 
 ```python
 from pyensembl import EnsemblRelease
@@ -160,25 +261,26 @@ With evidence `None` (every catalogue ORF), each assembly gets:
 | Genome | ORF source | Records | Placed | Not placed: main reasons |
 | --- | --- | ---: | ---: | --- |
 | Human Ensembl 115, GRCh38 | bundled | 7,264 | 7,239 | 24 splice paths changed |
-| Human Ensembl 75, GRCh37 | lifted, 1 hop | 7,247 | 6,857 | 271 transcripts absent, 108 paths changed |
-| Human Ensembl 54, NCBI36 | lifted, 1 hop | 7,253 | 4,195 | 2,710 transcripts absent |
-| Mouse Ensembl 115, GRCm39 | OpenProt | 2,976 | 2,854 | 109 without sequence |
-| Mouse Ensembl 102, GRCm38 | lifted, 1 hop | 2,975 | 2,856 | 112 without sequence |
-| Mouse Ensembl 67, NCBIM37 | lifted, 2 hops | 2,922 | 1,620 | 1,187 transcripts absent |
+| Human Ensembl 75, GRCh37 | lifted, 1 hop | 7,247 | 6,857 | 270 transcripts absent, 108 paths changed, 1 on a transcript with a different protein |
+| Human Ensembl 54, NCBI36 | lifted, 1 hop | 7,253 | 4,195 | 2,708 transcripts absent, 2 on a transcript with a different protein |
+| Human Ensembl rapid 2022_07, T2T-CHM13v2.0 | lifted, 1 hop | 7,208 | 6,767 | 383 on a transcript with a different protein, 58 on no transcript, 56 not lifted |
+| Mouse Ensembl 115, GRCm39 | OpenProt, Chang et al. and Yang et al. | 48,805 | 48,577 | 110 transcripts absent, 109 without sequence |
+| Mouse Ensembl 102, GRCm38 | all three lifted, 1 hop | 48,800 | 48,169 | 434 transcripts absent, 5 not lifted |
+| Mouse Ensembl 67, NCBIM37 | all three lifted, 2 hops | 48,642 | 38,105 | 9,217 transcripts absent, 163 not lifted |
 
 Older annotations lack many of the transcripts the sources name, so fewer ORFs
 place; the rest, and any that didn't lift, are listed with the reason, never
 silently dropped.
 
-- **Which catalogues.** Each catalogue contributes one release: the one on the
-  requested assembly if it has one, else its default, lifted. Each assembly
-  the pinned chains reach contributes one catalogue (a bundled one if any);
-  two candidates on one assembly raise rather than being chosen between.
-  Catalogues from several assemblies are merged, the requested assembly's
-  first, then by fewest hops. A source already used from a nearer assembly,
-  such as one catalogue registered again under another name on another
-  assembly, counts once. This depends only on the registry and the pinned
-  chains, never on what happens to be installed.
+- **Which catalogues.** Every catalogue for the species is used, merged: on
+  the requested assembly as is, on the others the pinned chains reach, lifted.
+  Each contributes one release: the one on the requested assembly if it has
+  one, else its default. The requested assembly's come first, then by fewest
+  hops; bundled first, then by the registry's priority, which decides whose ID
+  a shared ORF keeps, so a new catalogue never renames existing ORFs. A source already used from a nearer assembly, such as one catalogue
+  registered again under another name on another assembly, counts once. This
+  depends only on the registry and the pinned chains, never on what happens
+  to be installed.
 - **What's missing.** Missing catalogues and chain files are reported together,
   each with the command that installs it; nothing is downloaded while
   loading. `cache_root` (or `--cache-root`) is where both are looked for.
@@ -193,7 +295,44 @@ silently dropped.
   through GRCm38. `lift_to(catalog, build)` lifts through it.
   `catalog_status()` lists, for each catalogue, the assemblies it reaches and
   whether their chains are downloaded.
-- **Cost.** Lifting happens on each load: about 1.6 s for the human catalogue.
+- **Cost.** Lifting takes a few seconds (about 2 s for the human catalogue to
+  GRCh37 or CHM13); a process reuses the result for the same installed
+  catalogues and chains.
+- **Unlifted ORFs.** `ExtraORFs.unlifted` lists the ORFs the evidence rule
+  selects that a lift left out; proteomes count them as `unlifted` and list
+  them among their exclusions. `ExtraORFs.excluded` holds only records of its
+  catalogue, which is what Varcode looks up.
+
+### T2T-CHM13
+
+PyEnsembl's `EnsemblRelease` has no T2T-CHM13 annotation, so orfdata pins
+Ensembl's rapid-release one (2022_07) and installs it on request, checking each
+file's size and SHA-256 (it needs `orfdata[pyensembl]`). The T2T Consortium's
+GRCh38-to-CHM13 chain is public domain (CC0):
+
+```sh
+orfdata genomes install t2t-chm13          # 143 MB of Ensembl files, plus about 1.1 GB of PyEnsembl index
+orfdata chains download GRCh38 T2T-CHM13v2.0
+orfdata proteome --genome t2t-chm13 --output chm13.fasta
+```
+
+```python
+from orfdata import Proteome, load_genome
+
+genome = load_genome("t2t-chm13")          # A PyEnsembl Genome named T2T-CHM13v2.0, offline
+proteome = Proteome.build(genome)          # 104,957 annotated proteins + 2,039 MS-supported ORFs
+```
+
+The bundled GRCh38 catalogue lifts automatically: 7,208 of 7,264 ORFs lift, and
+6,767 place on CHM13 transcripts with their exact coding path and protein.
+Ensembl gave CHM13 transcripts new IDs, so placement is by path. Of the 441 that
+don't place, 383 lie on a CHM13 transcript with their exact coding path whose
+sequence, from CHM13's own alleles, encodes a different protein (in 15, the
+start codon itself differs); the exclusion reason names that transcript. The
+other 58 are on no CHM13 transcript. The genome keeps its species and assembly
+accession when pickled, for multiprocessing and Varcode. Your own CHM13 genome
+works too if its `reference_name` is `T2T-CHM13v2.0` and it has a `species`
+(or you pass `context=`); `hs1` or `chm13` get an error saying so.
 
 ## Lifting a catalogue by hand
 
@@ -235,11 +374,12 @@ print(len(grch37.records), len(grch37.metadata["liftover"]["unmapped"]), grch37.
 7247 17 orfdata-catalog:2026-10-11.1@GRCh37
 ```
 
-- **Pinned chain files.** Ensembl's human GRCh38 to and from
+- **Pinned chain files.** The T2T Consortium's GRCh38 to and from
+  [T2T-CHM13v2.0](https://github.com/marbl/CHM13) are public domain. Ensembl's human GRCh38 to and from
   [GRCh37 and NCBI36](https://ftp.ensembl.org/pub/assembly_mapping/homo_sapiens/),
   and mouse GRCm38 to and from
   [NCBIM37](https://ftp.ensembl.org/pub/assembly_mapping/mus_musculus/),
-  have no usage restrictions. `orfdata chains list` shows all eight. UCSC's mouse
+  have no usage restrictions. `orfdata chains list` shows all ten. UCSC's mouse
   [`mm10ToMm39`](https://hgdownload.soe.ucsc.edu/goldenPath/mm10/liftOver/README.txt)
   and [`mm39ToMm10`](https://hgdownload.soe.ucsc.edu/goldenPath/mm39/liftOver/README.txt)
   (GRCm38 and GRCm39) are free only for non-commercial use, so downloading
@@ -285,7 +425,7 @@ print(len(grch37.records), len(grch37.metadata["liftover"]["unmapped"]), grch37.
   2,186 MS-supported ORFs: 2,097 place, and 3 of the 89 excluded didn't lift.
 - **Size.** Each lifted record carries its original, so a lifted catalogue is
   about twice the size of its input (33 MB for the bundled one as JSON).
-- **Errors.** When no pinned chains reach an assembly, such as T2T-CHM13,
+- **Errors.** When no pinned chains reach an assembly, such as HG002,
   `load_catalog(species=..., genome_build=...)` gives the `orfdata lift
   --chain` command and the Python call that lifts with your own chain file.
 
@@ -306,7 +446,10 @@ use separate locations.
 
 ## Defining a release
 
-A registry definition maps a name to `default_version` and `versions`. Each
+A registry definition maps a name to `default_version`, `versions` and an
+optional integer `priority` (default 100): when catalogues are merged, lower
+priorities come first, and the first catalogue's ID names an ORF several
+share. Each
 version contains a `CatalogManifest`, `distribution` access terms and an
 `assets` entry for raw `catalog.json.gz` bytes, pinned by URL, SHA-256 and size.
 The manifest declares references, original `SourceAsset` inputs, citations,

{orfdata-0.27.1 → orfdata-0.28.0}/docs/coverage.md

@@ -31,9 +31,10 @@ Calibration and uncertainty remain attached to observations:
 ## ORF evidence and ribosome profiling
 
 The separate existence-evidence catalogue contains 7,264 human noncanonical
-ORFs, including 2,186 with reported MS support. Mouse has 2,976 OpenProt 2.2
-ORFs, each with at least two unique MS peptides, which you build from
-OpenProt's files; dog and yeast have no experimental ncORF catalogue yet. Its
+ORFs, including 2,186 with reported MS support. Mouse has 48,805 ORFs: Ribo-seq ORFs
+from Chang et al. and Yang et al., and 2,976 OpenProt 2.2 ORFs with at least
+two unique MS peptides, built from their sources' files; dog and yeast have no experimental
+ncORF catalogue yet. Its
 ribosome evidence and pooled PSM counts are not sample-level ribosome
 occupancy or protein abundance.
 

{orfdata-0.27.1 → orfdata-0.28.0}/docs/index.md

@@ -53,12 +53,13 @@ orfdata proteome --species human --release 115 --output human-e115.fasta
 
 | `evidence=` | Selects ORFs with | Human ORFs | Encoded in Ensembl 115 |
 | --- | --- | ---: | ---: |
-| `"ribosome"` | Ribosome profiling (all bundled ORFs) | 7,264 | 7,239 |
+| `"ribosome"` or `"riboseq"` | Ribosome profiling (all bundled ORFs) | 7,264 | 7,239 |
 | `"ms"` (default) | Any supporting MS peptide | 2,186 | 2,181 |
-| `"hla"` | Peptides presented by HLA molecules | 1,881 | 1,879 |
-| `"shotgun"` | Peptides from protease-digest MS | 73 | 72 |
+| `"immunopeptidomics"` | Peptides eluted from MHC (in human, HLA) molecules | 1,881 | 1,879 |
+| `"proteomics"` | Peptides from digested cellular proteins | 73 | 72 |
+| `"all"` | Every catalogue ORF | 7,264 | 7,239 |
 
-Most MS support in the bundled human catalogue comes from HLA
+Most MS support in the bundled human catalogue comes from
 immunopeptidomics. A presented peptide shows that the ORF was translated; it
 does not establish a stable protein. See
 [building a reference proteome](proteome.md#choose-the-evidence) for
@@ -95,9 +96,9 @@ transcript alongside them.
 
 | Species | Annotated proteins | Noncanonical ORFs |
 | --- | --- | --- |
-| Human | From PyEnsembl | 7,264 GRCh38 ORFs, bundled; lifted to GRCh37 and NCBI36 |
-| Mouse | From PyEnsembl | 2,976 GRCm39 ORFs from OpenProt 2.2, [built locally](catalogs.md#mouse-orfs-from-openprot); lifted to GRCm38 and NCBIM37 |
-| Dog | From PyEnsembl | None released; no qualified source yet |
+| Human | From PyEnsembl; T2T-CHM13 from a pinned Ensembl annotation | 7,264 GRCh38 ORFs, bundled; lifted to GRCh37, NCBI36 and [T2T-CHM13](catalogs.md#t2t-chm13) |
+| Mouse | From PyEnsembl | 48,805 ORFs on GRCm39: Ribo-seq ORFs of Chang et al. and Yang et al., and 2,976 MS-supported ones (OpenProt 2.2), [built locally](catalogs.md#mouse) with `orfdata setup --species mouse`; lifted to GRCm38 and NCBIM37 |
+| Dog | From PyEnsembl | None: no experimentally supported dog ORF set exists |
 | Budding yeast | From PyEnsembl | None released |
 
 Other assemblies use a catalogue lifted to them once `orfdata chains download`

{orfdata-0.27.1 → orfdata-0.28.0}/docs/proteome.md

@@ -52,11 +52,12 @@ the bundled human catalogue (dataset `2026-10-11.1`) and Ensembl 115:
 
 | Selection | Human ORFs | Encoded in Ensembl 115 |
 | --- | ---: | ---: |
-| `evidence=None` or `"ribosome"`: translated in ribosome profiling | 7,264 | 7,239 |
+| `evidence=None` or `"all"`: every catalogue ORF | 7,264 | 7,239 |
+| `evidence="ribosome"` or `"riboseq"`: translated in ribosome profiling | 7,264 | 7,239 |
 | `evidence="ms"` (default): any supporting MS peptide | 2,186 | 2,181 |
-| `evidence="hla"` or `"mhc"`: peptides presented by HLA | 1,881 | 1,879 |
+| `evidence="immunopeptidomics"`: peptides eluted from MHC (in human, HLA) molecules | 1,881 | 1,879 |
 | `evidence="ms_unclassified"`: MS with an unreported assay type | 537 | 535 |
-| `evidence="shotgun"`: protease-digest MS | 73 | 72 |
+| `evidence="proteomics"`: peptides from digested cellular proteins | 73 | 72 |
 | `"ms"` with `require_unique=True` | 1,623 | 1,621 |
 | `"ms"` with `require_reviewed=True` | 44 | 44 |
 
@@ -64,19 +65,28 @@ All bundled ORFs come from ribosome-profiling catalogues, so `"ribosome"`
 selects all of them. An MS-supported ORF can have peptides from several assay
 types. `require_unique` keeps peptides that map to one protein *within the MS
 source's search*, which is not uniqueness across every possible protein; in
-this catalogue only the author HLA mappings report it. `require_reviewed`
+this catalogue only the author HLA ligand mappings report it. `require_reviewed`
 keeps peptides with convincing community spectrum reviews. Because those two
 properties come from different sources, no bundled peptide has both.
 `require_resolved` excludes support that consists only of aggregate counts.
 See [ORFs and evidence](reference.md#evidence) for the full definitions.
 
+A list (or tuple or set) of names selects ORFs with any of them:
+`evidence=["proteomics", "immunopeptidomics"]` keeps ORFs with either kind of
+MS peptide, unlike `"ms"`, which also counts unclassified MS. For ORFs that
+need two kinds of evidence at once, combine rules with `&`, below.
+
 ### Separate Ribo-seq and MS stringency
 
 For more control, pass a rule. `RiboSeq(min_studies=n)` needs ribosome
 profiling in at least `n` independent studies the source names, and
-`MS(min_peptides=n, unique=..., reviewed=..., methods=...)` needs at least `n`
-distinct supporting peptide sequences. `&` requires both rules and `|` accepts
-either:
+`MS(min_peptides=n, unique=..., reviewed=..., mhc=...)` needs at least `n`
+distinct supporting peptide sequences. `MS(mhc=True)` counts only peptides
+eluted from MHC molecules (the immunopeptidome) and `MS(mhc=False)` only
+peptides from digested cellular proteins (the whole proteome). `MS()` counts
+both, and the MS peptides whose source doesn't say which. `methods=` names the
+kinds directly, for combinations such as `("proteomics", "ms_unclassified")`.
+`&` requires both rules and `|` accepts either:
 
 ```python
 from orfdata import MS, RiboSeq
@@ -84,7 +94,8 @@ from orfdata import MS, RiboSeq
 annotated.with_extra_orfs(RiboSeq(min_studies=2))                        # replicated Ribo-seq
 annotated.with_extra_orfs(RiboSeq(min_studies=2) & MS())                 # Ribo-seq and MS
 annotated.with_extra_orfs(RiboSeq(min_studies=2) | MS())                 # either
-annotated.with_extra_orfs(MS(methods=("shotgun", "ms_unclassified")))     # not HLA-only
+annotated.with_extra_orfs(RiboSeq() & MS(mhc=True))                     # Ribo-seq and an MHC ligand
+annotated.with_extra_orfs(MS(methods=("proteomics", "ms_unclassified")))  # not immunopeptidomics-only
 ```
 
 | Rule | Human ORFs | Encoded in Ensembl 115 |
@@ -92,23 +103,26 @@ annotated.with_extra_orfs(MS(methods=("shotgun", "ms_unclassified")))     # not
 | `RiboSeq(min_studies=2)` | 3,085 | 3,072 |
 | `RiboSeq(min_studies=3)` | 1,675 | 1,671 |
 | `MS(min_peptides=2)` | 682 | 681 |
-| `MS(methods=("shotgun", "ms_unclassified"))` | 610 | 607 |
+| `MS(methods=("proteomics", "ms_unclassified"))` | 610 | 607 |
 | `RiboSeq(min_studies=2) & MS()` | 1,296 | 1,293 |
 | `RiboSeq(min_studies=2) & MS(min_peptides=2)` | 457 | 456 |
 | `RiboSeq(min_studies=2) \| MS()` | 3,975 | 3,960 |
 
 GENCODE's `c…riboseqorf…` IDs are the 3,085 ORFs seen in two or more studies.
-Each method name above is the simplest rule: `"ms"` is `MS()`, `"hla"` is
-`MS(methods=("hla",))` and `"ribosome"` is `RiboSeq()`. A peptide counts once
+Each method name above is the simplest rule: `"ms"` is `MS()`,
+`"immunopeptidomics"` is `MS(mhc=True)`, `"proteomics"` is `MS(mhc=False)` and
+`"ribosome"` (or `"riboseq"`) is `RiboSeq()`. The names before 0.28.0 (`"shotgun"`, and `"mhc"` or `"hla"`)
+still work, with a deprecation warning, and catalogues that use them still
+load. A peptide counts once
 however many assay types report it, and so do peptides that differ only by I
-and L (equal in mass). Nested peptides, such as HLA length variants, count as
+and L (equal in mass). Nested peptides, such as MHC ligand length variants, count as
 one: the count is the largest set of supporting peptides none of which lies
 inside another, so a long peptide never hides two shorter, separate ones, and
 a stricter rule always selects a subset of a looser one. Of the 916 ORFs with
 two or more distinct peptide sequences, 234 have fewer than two that aren't
 nested. Each rule in `&` is checked on its own:
-`MS(methods="shotgun") & MS(unique=True)` needs a shotgun peptide and a unique
-peptide, not one unique shotgun peptide. A source's aggregate count stands for one observation:
+`MS(mhc=False) & MS(unique=True)` needs a proteomics peptide and a
+unique peptide, not one unique proteomics peptide. A source's aggregate count stands for one observation:
 it satisfies `RiboSeq()` and `MS()` but never `min_studies` or `min_peptides`
 above 1, `unique`, `reviewed` or `resolved=True`. The layer stores the rule,
 so a snapshot records exactly how its ORFs were chosen; `EvidenceRule.from_dict`
@@ -154,8 +168,8 @@ identical sequences, for example before an exact peptide search.
 
 Without `catalog=`, `load_catalog` picks the catalogue for the genome's species
 and assembly: a bundled one, or else one installed through
-[`CatalogRegistry`](catalogs.md). Human GRCh38 has the bundled one, and mouse
-GRCm39 has OpenProt's once [built](catalogs.md#mouse-orfs-from-openprot); for
+[`CatalogRegistry`](catalogs.md), several merged. Human GRCh38 has the bundled
+one, and mouse GRCm39 three once [installed](catalogs.md#mouse); for
 other species the error names the catalogues that exist. Pass `catalog=` to use
 another, such as `load_catalog(path)`. A catalogue for a different species or
 assembly is rejected.
@@ -187,7 +201,7 @@ other.
   differently counts once when their `ribosome_study_pmids` metadata give it
   the same PMID, as the bundled catalogue's does; without PMIDs it counts
   twice toward `RiboSeq(min_studies=...)`.
-- **Lookups.** Queries by transcript or gene, `for_transcript` and
+- **Lookups.** Queries by transcript, gene or biotype, `for_transcript` and
   `ExtraORFs.for_gene(..., source=True)` match every source's labels, and an
   ORF is on its source transcript when it is on any source's.
 - **Annotation pins.** Sources pinned to different annotation releases of the
@@ -288,15 +302,15 @@ protein ID. Fields that apply follow as `key=value`:
 
 ```text
 >ENSP00000269305.4 kind=annotated taxon=9606 source=ensembl:115 gene=ENSG00000141510 gene_name=TP53 transcript=ENST00000269305.9 contig=17 transcript_biotype=protein_coding
->c7riboseqorf56 kind=extra_orf taxon=9606 source=orfdata-catalog:2026-10-11.1 gene=ENSG00000169902 gene_name=TPST1 transcript=ENST00000304842.6 contig=7 transcript_biotype=protein_coding orf_type=uORF evidence=ribosome,mhc transcripts=18
->c10riboseqorf115 kind=extra_orf taxon=9606 source=orfdata-catalog:2026-10-11.1 gene=ENSG00000099204 gene_name=ABLIM1 transcript=ENST00000369253.6 contig=10 transcript_biotype=protein_coding orf_type=intORF evidence=ribosome,mhc source_transcript=ENST00000277895 transcripts=10
+>c7riboseqorf56 kind=extra_orf taxon=9606 source=orfdata-catalog:2026-10-11.1 gene=ENSG00000169902 gene_name=TPST1 transcript=ENST00000304842.6 contig=7 transcript_biotype=protein_coding orf_type=uORF evidence=ribosome,immunopeptidomics transcripts=18
+>c10riboseqorf115 kind=extra_orf taxon=9606 source=orfdata-catalog:2026-10-11.1 gene=ENSG00000099204 gene_name=ABLIM1 transcript=ENST00000369253.6 contig=10 transcript_biotype=protein_coding orf_type=intORF evidence=ribosome,immunopeptidomics source_transcript=ENST00000277895 transcripts=10
 ```
 
 For extra ORFs, the accession is the source ORF ID, `transcript` is its host
 transcript in this genome, and `gene`, `gene_name` and `transcript_biotype`
 describe that transcript in this genome. `orf_type` is the source's ORF
 category, kept verbatim; see [ORF categories](reference.md#orf-categories).
-`evidence` lists the supporting evidence types; `mhc` includes HLA.
+`evidence` lists the supporting evidence types.
 `transcripts` counts the encoding transcripts when there are several, and
 `source_transcript` names the source's transcript when the header's transcript
 is a different one, as for ABLIM1 above, whose source transcript is gone. See
@@ -319,7 +333,7 @@ options. Rules have their own options:
 ```sh
 orfdata proteome --species human --release 115 --min-ribo-seq-studies 2 --min-ms-peptides 1
 orfdata proteome --species human --release 115 --min-ribo-seq-studies 3 --min-ms-peptides 2 --either
-orfdata proteome --species human --release 115 --min-ms-peptides 1 --ms-methods shotgun,ms_unclassified
+orfdata proteome --species human --release 115 --min-ms-peptides 1 --ms-methods proteomics,ms_unclassified
 ```
 
 Given both a Ribo-seq and an MS requirement, an ORF needs both unless
@@ -332,8 +346,8 @@ JSON summary of the layers, identity and excluded ORFs goes to standard error.
 
 ## Not yet included
 
-Mouse noncanonical ORFs come from OpenProt once you build that catalogue; see
-[Mouse ORFs from OpenProt](catalogs.md#mouse-orfs-from-openprot). Dog
+Mouse noncanonical ORFs come from Chang et al.'s and Yang et al.'s Ribo-seq and
+OpenProt's MS catalogues once you install them; see [Mouse](catalogs.md#mouse). Dog
 noncanonical ORFs, and translated endogenous retrovirus (ERV) ORFs without
 annotated transcripts, need new source data; see the
 [data roadmap](https://github.com/openvax/orfdata/issues/3). Products from

{orfdata-0.27.1 → orfdata-0.28.0}/docs/reference.md

@@ -167,10 +167,12 @@ The host is one of possibly several transcripts that carry the same ORF; see
 
 ## Evidence
 
-Use `orf.evidence.has_evidence("ribosome")`, `"shotgun"` (protease-digest MS),
-`"mhc"` (including human HLA), or `"ms"` (any supporting MS observation).
-`"hla"` selects human HLA observations specifically. Source observations whose
-MS assay type is unspecified remain labelled `"ms_unclassified"`.
+Use `orf.evidence.has_evidence("ribosome")` (or `"riboseq"`), `"proteomics"` (peptides from
+digested cellular proteins), `"immunopeptidomics"` (peptides eluted from MHC
+molecules; HLA in human), or `"ms"` (any supporting MS observation). Source
+observations whose MS assay type is unspecified remain labelled
+`"ms_unclassified"`. Catalogues from before 0.28.0 say `"shotgun"`, `"mhc"`
+and `"hla"`; they load with the current names.
 
 ```python
 orf.evidence.has_evidence("ms", require_unique=True, require_reviewed=True)
@@ -302,17 +304,18 @@ another annotation release of the same species, strain and assembly is also
 accepted. ORFs on another assembly can be
 [lifted over](catalogs.md#every-assembly-of-a-species).
 
-`load_catalog(species="mouse")` returns a species' catalogue: the bundled one,
-or else the one downloaded or built through [`CatalogRegistry`](catalogs.md),
-such as the [OpenProt mouse catalogue](catalogs.md#mouse-orfs-from-openprot).
-`genome_build=` chooses between assemblies. It never downloads, and for a
+`load_catalog(species="mouse")` returns a species' ORFs: the bundled
+catalogue, or those downloaded or built through
+[`CatalogRegistry`](catalogs.md), merged when there are several, as mouse's
+[three](catalogs.md#mouse) are. `genome_build=` chooses the assembly. It never downloads, and for a
 species without a catalogue the error lists the catalogues that exist.
 
 `load_catalog(path)` reads a custom versioned JSON catalogue. Each catalogue is
 scoped to one species and assembly; each model records its NCBI taxon, scientific
 name, assembly, translation table and source. Human evidence is never transferred
-to a mouse ortholog. Mouse ORFs come from OpenProt, and their bindings are
-tested with real OpenProt rows on real Ensembl 106 transcripts.
+to a mouse ortholog. Mouse ORFs come from OpenProt, Chang et al. and Yang et
+al., and each importer is tested with real source rows on real Ensembl
+transcripts.
 
 To build a catalogue, create `ORFModel` objects from source coordinates and
 sequences, attach source observations with `ORFRecord.from_model(model, evidence)`,
@@ -320,10 +323,9 @@ and construct `ORFCatalog(records, metadata)`. Its `to_dict()` produces the
 `orfdata.catalog.v1` JSON representation. Genomic coding blocks are 1-based,
 inclusive, sorted by genomic position, and include the terminal stop codon.
 
-[OpenProt](https://openprot.org) supplies the mouse catalogue through a pinned
-importer that checks every row's coordinates and protein; see
-[Mouse ORFs from OpenProt](catalogs.md#mouse-orfs-from-openprot). Its human
-and yeast files are not imported yet.
+The mouse catalogues come from pinned importers that check every row's
+coordinates and protein; see [Mouse](catalogs.md#mouse). OpenProt's human and
+yeast files are not imported yet.
 
 ## Rebuild and verify