orfdata 0.29.0__tar.gz → 0.30.0__tar.gz

{orfdata-0.29.0/orfdata.egg-info → orfdata-0.30.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: orfdata
-Version: 0.29.0
+Version: 0.30.0
 Summary: Reference proteomes with evidence-backed noncanonical ORFs on PyEnsembl genomes
 Author: OpenVax
 License-Expression: Apache-2.0
@@ -88,14 +88,23 @@ immunopeptidomics: peptides eluted from HLA molecules.
 
 ## Species
 
-The bundled noncanonical ORF catalogue is human GRCh38. Mouse has 48,805:
-Ribo-seq ORFs of Chang et al. 2025 and Yang et al. 2024, and 2,976 MS-supported
-ones from OpenProt 2.2;
-`pip install "orfdata[data,build-data]"` then `orfdata setup --species mouse`
-builds all three from their sources' own files. After `orfdata chains download` (add
+| Species | Noncanonical ORFs | Sources |
+| --- | ---: | --- |
+| Human | 7,264 bundled; 57,607 with `catalogs="all"` | GENCODE; optionally Chang et al. 2025 and Yang et al. 2024 Ribo-seq |
+| Mouse | 48,805 | Chang et al. and Yang et al. Ribo-seq, OpenProt 2.2 MS |
+| Zebrafish | 9,759 | Yang et al. Ribo-seq |
+| C. elegans | 2,849 | Yang et al. Ribo-seq |
+| Yeast | 1,128 | Yang et al. Ribo-seq |
+| Dog | none: no public dog Ribo-seq or ncORF MS exists | |
+
+Human's bundled catalogue needs no setup. For the others, `pip install
+"orfdata[data,build-data]"`, then `orfdata setup --species mouse` (or
+`zebrafish`, `worm`, `yeast`, or `human --catalogs all`) builds the catalogues
+from their sources' own files ([details](https://openvax.github.io/orfdata/catalogs/)).
+After `orfdata chains download` (add
 `--accept-terms` for UCSC's non-commercial mouse chains), older assemblies
 (GRCh37, NCBI36, GRCm38, NCBIM37) and T2T-CHM13 use these catalogues lifted
-to them; `orfdata genomes install t2t-chm13` installs a CHM13 annotation. Dog and yeast proteomes contain their
+to them; `orfdata genomes install t2t-chm13` installs a CHM13 annotation. Dog proteomes contain their
 PyEnsembl-annotated proteins. `load_catalog(species=...)` and
 `with_extra_orfs()` pick the catalogue for a species and raise when none is
 available or built.

{orfdata-0.29.0 → orfdata-0.30.0}/README.md

@@ -56,14 +56,23 @@ immunopeptidomics: peptides eluted from HLA molecules.
 
 ## Species
 
-The bundled noncanonical ORF catalogue is human GRCh38. Mouse has 48,805:
-Ribo-seq ORFs of Chang et al. 2025 and Yang et al. 2024, and 2,976 MS-supported
-ones from OpenProt 2.2;
-`pip install "orfdata[data,build-data]"` then `orfdata setup --species mouse`
-builds all three from their sources' own files. After `orfdata chains download` (add
+| Species | Noncanonical ORFs | Sources |
+| --- | ---: | --- |
+| Human | 7,264 bundled; 57,607 with `catalogs="all"` | GENCODE; optionally Chang et al. 2025 and Yang et al. 2024 Ribo-seq |
+| Mouse | 48,805 | Chang et al. and Yang et al. Ribo-seq, OpenProt 2.2 MS |
+| Zebrafish | 9,759 | Yang et al. Ribo-seq |
+| C. elegans | 2,849 | Yang et al. Ribo-seq |
+| Yeast | 1,128 | Yang et al. Ribo-seq |
+| Dog | none: no public dog Ribo-seq or ncORF MS exists | |
+
+Human's bundled catalogue needs no setup. For the others, `pip install
+"orfdata[data,build-data]"`, then `orfdata setup --species mouse` (or
+`zebrafish`, `worm`, `yeast`, or `human --catalogs all`) builds the catalogues
+from their sources' own files ([details](https://openvax.github.io/orfdata/catalogs/)).
+After `orfdata chains download` (add
 `--accept-terms` for UCSC's non-commercial mouse chains), older assemblies
 (GRCh37, NCBI36, GRCm38, NCBIM37) and T2T-CHM13 use these catalogues lifted
-to them; `orfdata genomes install t2t-chm13` installs a CHM13 annotation. Dog and yeast proteomes contain their
+to them; `orfdata genomes install t2t-chm13` installs a CHM13 annotation. Dog proteomes contain their
 PyEnsembl-annotated proteins. `load_catalog(species=...)` and
 `with_extra_orfs()` pick the catalogue for a species and raise when none is
 available or built.

{orfdata-0.29.0 → orfdata-0.30.0}/docs/catalogs.md

@@ -16,11 +16,19 @@ original PyEnsembl transcript through `ReferenceORF`. To use several catalogues
 for one species together, see
 [merging catalogues](proteome.md#merging-catalogues).
 
-The packaged registry has three mouse catalogues, each built from its sources'
-own files: [OpenProt's MS-supported ORFs](#mouse-orfs-from-openprot),
-[Chang et al.'s Ribo-seq ORFs](#mouse-ribo-seq-orfs-chang-et-al) and
-[Yang et al.'s RibORF ORFs](#mouse-riborf-orfs-yang-et-al). `orfdata setup
---species mouse` installs all three, with the chain files. `orfdata catalogs status`
+The packaged registry builds each catalogue from its sources' own files:
+
+| Species | Catalogues | ORFs |
+| --- | --- | ---: |
+| Mouse | [OpenProt's MS-supported ORFs](#mouse-orfs-from-openprot), [Chang et al.'s](#mouse-ribo-seq-orfs-chang-et-al) and [Yang et al.'s](#mouse-riborf-orfs-yang-et-al) Ribo-seq ORFs | 48,805 merged |
+| Human | the bundled GENCODE set; optionally [Chang et al.'s and Yang et al.'s Ribo-seq ORFs](#human-ribo-seq-orfs-optional) | 7,264; 57,607 with both |
+| Zebrafish | [Yang et al.'s Ribo-seq ORFs](#zebrafish-c-elegans-and-yeast) | 9,759 |
+| C. elegans | [Yang et al.'s Ribo-seq ORFs](#zebrafish-c-elegans-and-yeast) | 2,849 |
+| Yeast | [Yang et al.'s Ribo-seq ORFs](#zebrafish-c-elegans-and-yeast) | 1,128 |
+| Dog | [none exists](#dog) | 0 |
+
+`orfdata setup --species mouse` (or zebrafish, worm, yeast) installs a
+species' catalogues, with the chain files. `orfdata catalogs status`
 (in Python, `catalog_status()`) lists every catalogue `load_catalog` can open,
 the bundled one included, with its species, assembly, record count, whether it
 is installed and whether it is used:
@@ -94,10 +102,7 @@ Ensembl 115 (GRCm39):
 
 Loading the merged catalogue takes about 10 seconds and 0.9 GB of memory, once
 per process, whichever evidence is selected. On GRCm38 and NCBIM37 all three
-are [lifted automatically](#every-assembly-of-a-species). Dog has no
-experimentally supported ORF catalogue from any source: OpenProt and the
-Ribo-seq resources have no dog data, and dog MS studies searched only canonical
-proteins.
+are [lifted automatically](#every-assembly-of-a-species).
 
 ### Mouse Ribo-seq ORFs (Chang et al.)
 
@@ -238,6 +243,114 @@ Cite Leblanc S, et al. *OpenProt 2.0 builds a path to the functional
 characterization of alternative proteins.* Nucleic Acids Res. 2024;52(D1):D522–D528,
 [doi:10.1093/nar/gkad1050](https://doi.org/10.1093/nar/gkad1050).
 
+## Human Ribo-seq ORFs (optional)
+
+Human's default is the bundled GENCODE catalogue: 7,264 ORFs, offline, with
+no setup. Two larger Ribo-seq catalogues are optional, used only when asked
+for, because they would change every human selection and make loading human
+ORFs about 60 times slower:
+
+```sh
+orfdata setup --species human --catalogs all   # Builds both; about 225 MB of sources
+```
+
+```python
+from orfdata import Proteome, extra_orfs, load_catalog
+
+catalog = load_catalog(species="human", catalogs="all")   # Bundled, Chang et al. and Yang et al., merged
+orfs = extra_orfs(genome, "ribosome", catalogs="all")
+proteome = Proteome.build(genome, "all", catalogs="all")
+load_catalog(species="human", catalogs=["ncorf-mammals-human"])   # Just one
+```
+
+`orfdata proteome --catalogs all` does the same. They merge after the bundled
+catalogue, Chang et al.'s before Yang et al.'s as for mouse, so an ORF in
+several keeps the first one's ID. Together they give 57,607 ORFs, of which 57,485 place on Ensembl 115; that
+takes about a minute and 3.2 GB of memory, against about a second for the
+bundled catalogue alone. Neither adds MS evidence, so `"ms"` selects the same
+ORFs either way.
+
+- **`ncorf-mammals-human`:** the human ncORFs of [Chang et
+  al.](#mouse-ribo-seq-orfs-chang-et-al), from 174 Ribo-seq libraries
+  (Table S1), built as for mouse on Ensembl 107 GRCh38: 11,621 of the
+  source's 11,623, two having a coding length that isn't their protein plus
+  a stop. Types: uORF (6,178), lncORF (2,231), uoORF (1,633), iORF (773),
+  dORF (694) and doORF (112); 997 start at GTG.
+- **`riborf-human`:** the human ORFs of [Yang et
+  al.](#mouse-riborf-orfs-yang-et-al), from 136 samples in 30 GEO series, on
+  GENCODE V39, placed through the same Ensembl 105 transcripts (GENCODE V39
+  is Ensembl 105): 51,174 of the 51,275 rows long enough; 101 are on
+  transcripts with no sequence in Ensembl 105's FASTAs. Types: uORF (23,664),
+  lncORF (11,379), uoORF (4,548), dORF (3,747), N-terminal truncation (3,561),
+  iORF (2,547), extension (1,046) and doORF (682).
+
+`catalogs` takes `None` (the species' default set), `"all"`, or catalogue
+names, `"bundled"` among them. `orfdata catalogs status` marks optional
+catalogues. Everywhere but human, every catalogue is in the default set.
+
+## Zebrafish, C. elegans and yeast
+
+```sh
+orfdata setup --species zebrafish   # or worm, or yeast
+```
+
+Yang et al.'s RibORF 2.0 ORFs of [the mouse catalogue's
+kind](#mouse-riborf-orfs-yang-et-al), at least 5 residues long, from each
+species' sheet of Supplementary Data 3, imported the same way:
+
+| Catalogue | Samples (GEO series) | Placed through | ORFs | On Ensembl 115 |
+| --- | --- | --- | ---: | ---: |
+| `riborf-zebrafish` | 20 (2) | Ensembl 115 GRCz11 transcripts | 9,759 | 9,759 |
+| `riborf-worm` | 39 (1) | Ensembl 105 WBcel235 transcripts | 2,849 | 2,433 |
+| `riborf-yeast` | 5 (1) | the R64-1-1 genome (sacCer3) | 1,128 | 171 |
+
+- **Zebrafish:** every source transcript has the same length in Ensembl 100
+  to 115, so the catalogue uses 115. 18 rows have ambiguous bases in the ORF
+  and are left out. Types: N-terminal truncation (3,805), uORF (3,543), dORF
+  (771), iORF (605), lncORF (411), uoORF (304), extension (224) and doORF (96).
+- **C. elegans:** the source's transcripts match Ensembl 100 and 105, and
+  WormBase changed many models after, so the catalogue uses 105; all 2,849
+  place there, 2,433 on 115. Transcript names with dots of their own
+  (`F45H11.11.1`) are kept whole. Types: truncation (614), iORF (596),
+  extension (556), uORF (521), uoORF (217), lncORF (151), doORF (117) and dORF
+  (77).
+- **Yeast:** the source's yeast transcripts carry UTRs, and some are unstable
+  transcripts (CUTs, SUTs), none of which Ensembl's yeast annotation has; its
+  transcripts span only the CDS. So each ORF is placed from its genomic span
+  and translated from the pinned genome. The source gives block sizes but not
+  starts, so the 4 ORFs on two exons are rebuilt from their first block's
+  size at the start and last block's at the end, each across a GT-AG intron;
+  Ty1's programmed frameshift (`YNL284C-B`, two blocks 1 nt apart) is left
+  out. A record names the Ensembl transcript (`YNL046W_mRNA`) only when that
+  transcript carries its coding path; otherwise it names the source's
+  transcript (`YNR004W`, `CUT808`), with the Ensembl gene where there is one.
+  Only the 171 ORFs on an Ensembl transcript, such as truncations and iORFs,
+  place on a PyEnsembl genome; the rest (uORFs in 5′ UTRs, ORFs on CUTs) stay
+  in the catalogue, queryable by region and gene, until ORFs on source-defined
+  RNAs are supported (#66). Types: uORF (321), lncORF (249), dORF (224),
+  truncation (137), extension (67), uoORF (55), doORF (40) and iORF (35).
+
+## Dog
+
+No public resource gives dog ORF-level translation evidence, as of October
+2026:
+
+- **Ribo-seq.** No dog ribosome profiling is in GEO, SRA or ENA, and none of
+  OpenProt, RiboSeq.org (147 organisms), RPFdb, SmProt or GWIPS-viz has dog.
+  The one dog experiment published, Pfeffer et al. 2015 (canine pancreas
+  rough-ER microsomes), deposited no reads.
+- **Mass spectrometry.** Every dog dataset searched canonical proteins only,
+  so none reports a noncanonical peptide as deposited: the Canine
+  PeptideAtlas (PXD052932, 245,884 peptides from 13 tissues, plasma and
+  urine), the dog tumour DLA immunopeptidome (PXD074485, about 12,700
+  ligands), the Pfizer TMT atlas (PXD070290) and the dog brain proteome
+  (PXD033420, PXD034449).
+
+Dog ncORF evidence therefore needs those spectra searched again against dog
+ORF sequences. Human and mouse ORFs are never transferred to dog by orthology.
+Dog proteomes contain their PyEnsembl-annotated proteins, and dog RNA and
+protein abundance are available as [measurements](measurements.md).
+
 ## Every assembly of a species
 
 Catalogues on another assembly of the species are lifted automatically. Download

{orfdata-0.29.0 → orfdata-0.30.0}/docs/coverage.md

@@ -33,8 +33,10 @@ Calibration and uncertainty remain attached to observations:
 The separate existence-evidence catalogue contains 7,264 human noncanonical
 ORFs, including 2,186 with reported MS support. Mouse has 48,805 ORFs: Ribo-seq ORFs
 from Chang et al. and Yang et al., and 2,976 OpenProt 2.2 ORFs with at least
-two unique MS peptides, built from their sources' files; dog and yeast have no experimental
-ncORF catalogue yet. Its
+two unique MS peptides, built from their sources' files. Yang et al.'s Ribo-seq ORFs
+cover zebrafish (9,759), C. elegans (2,849) and yeast (1,128), and with Chang et al.'s
+are optional human catalogues; dog has no experimental ncORF catalogue, because no
+dog Ribo-seq or ncORF MS search is public ([details](catalogs.md#dog)). Its
 ribosome evidence and pooled PSM counts are not sample-level ribosome
 occupancy or protein abundance.
 

{orfdata-0.29.0 → orfdata-0.30.0}/docs/index.md

@@ -96,10 +96,12 @@ transcript alongside them.
 
 | Species | Annotated proteins | Noncanonical ORFs |
 | --- | --- | --- |
-| Human | From PyEnsembl; T2T-CHM13 from a pinned Ensembl annotation | 7,264 GRCh38 ORFs, bundled; lifted to GRCh37, NCBI36 and [T2T-CHM13](catalogs.md#t2t-chm13) |
+| Human | From PyEnsembl; T2T-CHM13 from a pinned Ensembl annotation | 7,264 GRCh38 ORFs, bundled; lifted to GRCh37, NCBI36 and [T2T-CHM13](catalogs.md#t2t-chm13). 57,607 with the [optional Ribo-seq catalogues](catalogs.md#human-ribo-seq-orfs-optional) of Chang et al. and Yang et al. (`catalogs="all"`) |
 | Mouse | From PyEnsembl | 48,805 ORFs on GRCm39: Ribo-seq ORFs of Chang et al. and Yang et al., and 2,976 MS-supported ones (OpenProt 2.2), [built locally](catalogs.md#mouse) with `orfdata setup --species mouse`; lifted to GRCm38 and NCBIM37 |
-| Dog | From PyEnsembl | None: no experimentally supported dog ORF set exists |
-| Budding yeast | From PyEnsembl | None released |
+| Zebrafish | From PyEnsembl | 9,759 GRCz11 Ribo-seq ORFs of Yang et al., [built locally](catalogs.md#zebrafish-c-elegans-and-yeast) |
+| C. elegans | From PyEnsembl | 2,849 WBcel235 Ribo-seq ORFs of Yang et al., built locally |
+| Budding yeast | From PyEnsembl | 1,128 R64-1-1 Ribo-seq ORFs of Yang et al., built locally; 171 lie on Ensembl's transcripts |
+| Dog | From PyEnsembl | None: [no public dog Ribo-seq or ncORF MS exists](catalogs.md#dog) |
 
 Other assemblies use a catalogue lifted to them once `orfdata chains download`
 has fetched the chain files (with `--accept-terms` for UCSC's non-commercial

{orfdata-0.29.0 → orfdata-0.30.0}/docs/species.md

@@ -4,7 +4,7 @@ Resolve names and IDs offline with `orfdata.reference.species_taxon(name)` and
 `species_name_for_taxon(taxon_id)`. The latter returns a canonical species name,
 or `None` for an unknown positive ID, without requiring an Ensembl annotation.
 
-ORFdata uses the same objects for human, mouse, dog, yeast and custom species.
+ORFdata uses the same objects for human, mouse, dog, zebrafish, C. elegans, yeast and custom species.
 Every ORF states its NCBI species taxon, assembly and genetic code. A
 `ReferenceContext` also pins the annotation provider/version and can retain a
 versioned assembly accession, strain taxon/label and sequence-set SHA-256.

{orfdata-0.29.0 → orfdata-0.30.0}/orfdata/catalog.py

@@ -917,16 +917,21 @@ def load_uorf_catalog(path=None, *, expected_sha256=None):
 _BUNDLED = {("homo_sapiens", "GRCh38"): "orf-evidence"}
 
 
-def load_catalog(path=None, *, species=None, genome_build=None, registry=None, expected_sha256=None):
+def load_catalog(path=None, *, species=None, genome_build=None, registry=None, expected_sha256=None,
+                 catalogs=None):
     """Load the ORF catalogue for a species and assembly, or a local catalogue file.
 
     ``species`` is a name such as ``"mouse"`` or ``"mus_musculus"``; as in
     PyEnsembl, it defaults to human. ``genome_build`` is needed only when
     catalogues cover several assemblies of the species.
 
-    Every catalogue for the species is used: the bundled ones and those in
+    The species' catalogues are used: the bundled ones and those in
     ``registry`` (by default the packaged :class:`CatalogRegistry`), each at
-    its release on ``genome_build`` if it has one, else its default. One on
+    its release on ``genome_build`` if it has one, else its default.
+    ``catalogs`` chooses which: ``None`` for the species' default set,
+    ``"all"`` to add the optional ones too (such as human's Ribo-seq
+    catalogues beside the bundled GENCODE set), or a name or list of names
+    (``"bundled"`` for the bundled one). One on
     another assembly is lifted to ``genome_build`` through the pinned chain
     files, in as many hops as needed, and all are merged. Catalogues and chain files must already
     be installed (``orfdata catalogs download``/``build``, ``orfdata chains
@@ -947,14 +952,17 @@ def load_catalog(path=None, *, species=None, genome_build=None, registry=None, e
             raise ValueError(f"{path} is a {_species(catalog.species)} catalogue, not {_species(species)}")
         if genome_build is not None and genome_build != catalog.genome_build:
             raise ValueError(f"{path} is a {catalog.genome_build} catalogue, not {genome_build}")
+        if catalogs is not None:
+            raise ValueError("Pass a catalogue path or catalogs, not both")
         return catalog
     from .liftover import chain_downloaded, lift_to
 
     species = _species("human" if species is None else species)
     choices = _catalog_choices(registry)
-    found = [c for c in choices if c.species == species]
+    found = _selected([c for c in choices if c.species == species], catalogs, species)
     builds = sorted({c.genome_build for c in found})
-    known = "; ".join(f"{c.species} {c.genome_build} ({c.label})" for c in choices) or "none"
+    known = "; ".join(f"{c.species} {c.genome_build} ({c.label})"
+                      for c in sorted(choices, key=lambda c: (c.species, c.genome_build, c.name))) or "none"
     annotated_only = (" For a proteome of annotated proteins only, pass extra_orfs=False "
                       "(orfdata proteome --extra-orf-evidence none).")
     if not found:
@@ -1002,8 +1010,10 @@ def load_catalog(path=None, *, species=None, genome_build=None, registry=None, e
         except ImportError:
             step = "orfdata[data] installed, then "
         accept = " --accept-terms" if any(c.restricted for c in missing) else ""
+        chosen = ("" if catalogs is None else " --catalogs " + (
+            catalogs if isinstance(catalogs, str) else ",".join(catalogs)))
         raise ValueError(f"{species} {genome_build} ORFs need {step}" + "; and ".join(needs)
-                         + f". Or install everything at once with 'orfdata setup --species {species}{accept}"
+                         + f". Or install everything at once with 'orfdata setup --species {species}{chosen}{accept}"
                          f"{cli_root}'")
     # Lifting and merging take seconds; one process reuses the result for exactly the same inputs.
     key = (species, genome_build, expected_sha256, tuple((c.name, c.version, c.resource, _installed_at(c), tuple(
@@ -1069,6 +1079,23 @@ def _assembly_hint(species, genome_build, choices):
     return f" {genome_build} is a {', '.join(sorted(others))} assembly, not {species}." if others else ""
 
 
+def _selected(found, catalogs, species):
+    """The species' catalogues a selection names: None for its default set, "all" for every one, or names."""
+    if catalogs is None:
+        return [c for c in found if not c.optional]
+    names = [catalogs] if isinstance(catalogs, str) else list(catalogs)
+    if "all" in names:
+        return found
+    known = sorted({c.name for c in found})
+    if not known:
+        return []  # load_catalog names the species that have catalogues.
+    unknown = [n for n in names if n not in known]
+    if not names or unknown:
+        raise ValueError(f"No {species} catalogue named {', '.join(map(repr, unknown)) or 'in an empty selection'}; "
+                         f"{species} has {', '.join(known)}, or pass catalogs='all'")
+    return [c for c in found if c.name in names]
+
+
 def _plan(found, species, genome_build):
     """The catalogues load_catalog uses for one species and assembly, each with the chains that lift it there.
 
@@ -1125,6 +1152,7 @@ class _Choice(NamedTuple):
     resource: Optional[str] = None
     version: Optional[str] = None
     priority: int = 0
+    optional: bool = False
 
     @property
     def label(self):
@@ -1137,9 +1165,10 @@ def _catalog_choices(registry):
 
     registry = CatalogRegistry() if registry is None else registry
     priority = getattr(registry, "priority", lambda name: 100)
+    optional = getattr(registry, "optional", lambda name: False)
     return [_Choice(s, b, "bundled", resource=r) for (s, b), r in _BUNDLED.items()] + [
         _Choice(_species(m.references[0].species), m.genome_build, m.catalog_id, registry, version=m.version,
-                priority=priority(m.catalog_id))
+                priority=priority(m.catalog_id), optional=optional(m.catalog_id))
         for m in registry.list()]
 
 
@@ -1209,7 +1238,8 @@ class CatalogStatus:
     """One ORF catalogue load_catalog can open, and whether it is ready offline.
 
     ``selected`` marks the ones ``load_catalog(species=..., genome_build=...)``
-    uses, merged, for their species and assembly. ``installed`` is ``"bundled"``,
+    uses, merged, for their species and assembly. ``optional`` marks those it
+    uses only when asked for (``catalogs="all"`` or by name). ``installed`` is ``"bundled"``,
     ``"downloaded"``, ``"built"``, ``"damaged"`` or None; ``acquire`` names the explicit step
     that installs a missing one, ``"download"`` or ``"build"``. ``lifts_to``
     lists the other assemblies the pinned chains reach, each with its chains
@@ -1226,6 +1256,7 @@ class CatalogStatus:
     installed: Optional[str]
     acquire: Optional[str]
     lifts_to: tuple = ()
+    optional: bool = False
 
 
 def catalog_status(*, registry=None):
@@ -1242,13 +1273,15 @@ def catalog_status(*, registry=None):
     root = getattr(registry, "_cache_root", None)
     plans = {}
 
-    def plan(species, build):
-        if (species, build) not in plans:
+    def plan(species, build, catalogs=None):
+        """What load_catalog uses on this assembly: the default set, or with ``catalogs="all"`` the optional too."""
+        if (species, build, catalogs) not in plans:
             try:
-                plans[species, build] = _plan([c for c in choices if c.species == species], species, build)
+                plans[species, build, catalogs] = _plan(
+                    _selected([c for c in choices if c.species == species], catalogs, species), species, build)
             except ValueError:
-                plans[species, build] = []
-        return plans[species, build]
+                plans[species, build, catalogs] = []
+        return plans[species, build, catalogs]
 
     def downloaded(chain):
         try:
@@ -1261,7 +1294,7 @@ def catalog_status(*, registry=None):
         selected = any(choice == c and not route for choice, route in plan(c.species, c.genome_build))
         lifts = tuple(dict(genome_build=build, chains=[x.name for x in route], downloaded=all(map(downloaded, route)))
                       for build, route in sorted(chain_routes(c.species, c.genome_build).items())
-                      if any(choice == c for choice, _ in plan(c.species, build)))
+                      if any(choice == c for choice, _ in plan(c.species, build, "all" if c.optional else None)))
         if c.registry is None:
             manifest = json.loads(files("orfdata").joinpath(f"data/{c.resource}/manifest.json").read_text())
             rows.append(CatalogStatus(c.species, c.genome_build, c.name, manifest["dataset_version"],
@@ -1271,5 +1304,6 @@ def catalog_status(*, registry=None):
         info = c.registry.info(c.name, c.version)
         rows.append(CatalogStatus(c.species, c.genome_build, c.name, c.version, info.title, info.coverage["records"],
                                   selected, _installed(c),
-                                  "download" if c.registry.published(c.name, c.version) else "build", lifts))
+                                  "download" if c.registry.published(c.name, c.version) else "build", lifts,
+                                  c.optional))
     return tuple(rows)

{orfdata-0.29.0 → orfdata-0.30.0}/orfdata/catalogs.py

@@ -57,6 +57,10 @@ def _builder(manifest):
         from .sources.riborf import import_orfs
 
         return lambda inputs, output: import_orfs(inputs, output, manifest)
+    if importer == "riborf-genome-v1":
+        from .sources.riborf import import_genome_orfs
+
+        return lambda inputs, output: import_genome_orfs(inputs, output, manifest)
     raise ValueError("No packaged importer for this catalogue release")
 
 
@@ -82,6 +86,8 @@ class CatalogRegistry:
                 raise ValueError("A catalogue default must be a pinned version")
             if type(entry.get("priority", 0)) is not int:
                 raise ValueError(f"The {name} catalogue's priority must be an integer, lower merging first")
+            if type(entry.get("optional", False)) is not bool:
+                raise ValueError(f"The {name} catalogue's optional flag must be true or false")
             for version, release in entry["versions"].items():
                 manifest = CatalogManifest.from_dict(release["manifest"])
                 if (manifest.catalog_id, manifest.version) != (name, version):
@@ -130,6 +136,12 @@ class CatalogRegistry:
             raise ValueError(f"Unknown catalogue {name!r}; see 'orfdata catalogs list' or CatalogRegistry().list()")
         return self._definitions[name].get("priority", 100)
 
+    def optional(self, name):
+        """Whether the catalogue is used only when asked for (``catalogs="all"`` or its name), not by default."""
+        if name not in self._definitions:
+            raise ValueError(f"Unknown catalogue {name!r}; see 'orfdata catalogs list' or CatalogRegistry().list()")
+        return self._definitions[name].get("optional", False)
+
     def published(self, name, version=None):
         """Whether the release has a published artifact to download; otherwise it is built locally."""
         version = self._version(name, version)

{orfdata-0.29.0 → orfdata-0.30.0}/orfdata/cli.py

@@ -59,13 +59,16 @@ def _setup(argv, prog):
     parser.add_argument("--accept-terms", action="store_true",
                         help="Accept usage terms of restricted chain files, such as UCSC's non-commercial ones")
     parser.add_argument("--force", action="store_true", help="Install again even what is already installed")
+    parser.add_argument("--catalogs", metavar="NAMES", type=_catalog_names,
+                        help="'all' to add the species' optional catalogues, such as human's Ribo-seq ones, or "
+                             "comma-separated catalogue names; the species' default set by default")
     parser.add_argument("--cache-root", help="Cache directory instead of ORFDATA_CACHE, OPENVAX_DATA_CACHE or the default")
     parser.add_argument("--timeout", type=float, default=60,
                         help="Seconds a download may stall before it is retried, then fails (default 60)")
     args = parser.parse_args(argv)
     try:
         result = install_species(args.species, accept_terms=args.accept_terms, force=args.force,
-                                 cache_root=args.cache_root, timeout=args.timeout)
+                                 cache_root=args.cache_root, timeout=args.timeout, catalogs=args.catalogs)
     except InstallError as error:  # What did install is still reported.
         json.dump(error.result, sys.stdout, indent=2)
         sys.stdout.write("\n")
@@ -76,6 +79,11 @@ def _setup(argv, prog):
     sys.stdout.write("\n")
 
 
+def _catalog_names(value):
+    """``--catalogs``: "all", or comma-separated catalogue names."""
+    return value if value == "all" else [name for name in value.split(",") if name]
+
+
 def _add_query_arguments(parser, *, catalog_path=True):
     if catalog_path:
         parser.add_argument("--catalog", action="append",