orfdata 0.35.0__tar.gz → 0.36.0__tar.gz

{orfdata-0.35.0/orfdata.egg-info → orfdata-0.36.0}/PKG-INFO

@@ -1,17 +1,20 @@
 Metadata-Version: 2.4
 Name: orfdata
-Version: 0.35.0
+Version: 0.36.0
 Summary: Reference proteomes with evidence-backed noncanonical ORFs on PyEnsembl genomes
 Author: OpenVax
 License-Expression: Apache-2.0
 Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 License-File: LICENSE
+Requires-Dist: datacache<2,>=1.25.1
+Requires-Dist: pandas<4,>=3
+Requires-Dist: pyyaml<7,>=6
 Provides-Extra: pyensembl
 Requires-Dist: pyensembl>=2.27.2; extra == "pyensembl"
 Provides-Extra: build-data
 Requires-Dist: openpyxl>=3.1; extra == "build-data"
-Requires-Dist: biopython<2,>=1.80; extra == "build-data"
+Provides-Extra: viral
 Provides-Extra: data
 Requires-Dist: datacache<2,>=1.25.1; extra == "data"
 Provides-Extra: columnar
@@ -22,9 +25,9 @@ Requires-Dist: ruff; extra == "dev"
 Requires-Dist: build; extra == "dev"
 Requires-Dist: twine; extra == "dev"
 Requires-Dist: openpyxl>=3.1; extra == "dev"
-Requires-Dist: biopython<2,>=1.80; extra == "dev"
 Requires-Dist: pyensembl>=2.27.2; extra == "dev"
 Requires-Dist: datacache<2,>=1.25.1; extra == "dev"
+Requires-Dist: gb-io<0.5,>=0.4; extra == "dev"
 Provides-Extra: docs
 Requires-Dist: mkdocs<2,>=1.6; extra == "docs"
 Requires-Dist: mkdocs-material<10,>=9.5; extra == "docs"
@@ -48,12 +51,37 @@ evidence.
 Use Python 3.11 or later. Install orfdata and one PyEnsembl annotation:
 
 ```sh
-python -m pip install 'orfdata[pyensembl,data,build-data]'
+python -m pip install 'orfdata[pyensembl,build-data]'
 pyensembl install --release 115 --species human
 # For the ncORF evidence presets and "all":
 orfdata setup --species human --catalogs all
 ```
 
+List every bundled, downloadable or buildable resource and what is installed:
+
+```sh
+orfdata download list
+orfdata download status
+```
+
+`orfdata setup --species human` also installs `gdc-viruses`, the default
+200-reference GDC viral screen for human transcriptome analysis;
+`--collections none` skips it. Viral support is part of the base install, and
+only the data is downloaded. To acquire only the viral references:
+
+```sh
+orfdata download viral --species human
+orfdata viral summary
+```
+
+Each viral collection keeps three layers apart — the viral genomes, one coding
+transcript per source CDS feature with a GTF placing it on them, and the
+proteins those transcripts encode. Because the three share identifiers, the set
+opens as a PyEnsembl genome, so varcode, isovar, topiary and vaxrank can
+annotate variants on viral contigs. `orfdata proteome --annotation DIR` writes
+the same three files for noncanonical ORFs. See
+[Downstream tools](https://openvax.github.io/orfdata/downstream/).
+
 ## Example
 
 ```python
@@ -140,6 +168,13 @@ amino acids, parent protein accession, sample, change IDs and source before
 `combined.with_derived_proteins(...)` adds them. These sequences are not
 inferred from variants or included in the reference proteome by default.
 
+For a named, checksummed sequence input with no genomic placement, use the
+general `ProteinSet`. `with_protein_set(...)` records a donor, transgene or
+other selected component; `with_viral(...)` uses the same input plus a sourced
+host association. ncORF sources with transcript coordinates and evidence stay
+as `ORFCatalog` records and produce the same `ProteinSequence` output when
+bound to a reference.
+
 ```sh
 orfdata proteome --species human --release 115 --nc-orf-sources all --viral all --output human-all-proteins.fasta
 ```
@@ -159,10 +194,14 @@ packaged bases and coding paths. The functional loaders remain available.
 When annotating an existing alignment, `viral_orfs(reference_dna,
 contig="HPV16")` checks its actual reference sequence before applying the same
 models.
-The first [coding reference](https://openvax.github.io/orfdata/viral/) contains
-the nine RefSeq NC_001526.4 HPV16 CDSs, with native and GDC placements;
-the host association is distinct from the virus's own taxon. Broader
-imports remain in [the viral roadmap](https://github.com/openvax/orfdata/issues/91).
+The curated [coding reference](https://openvax.github.io/orfdata/viral/)
+contains the nine RefSeq NC_001526.4 HPV16 CDSs, with native and GDC
+placements; the host association is distinct from the virus's own taxon. The
+separate `gdc-viruses` source bundle contains all 200 GDC source genomes and
+their annotated CDS/protein FASTAs for human analysis. It does not assert a
+human source host for every virus; source-host metadata remains separate.
+Promoting further source records into the curated
+`ViralReference` API remains in the [viral roadmap](https://github.com/openvax/orfdata/issues/91).
 
 ## Measurement datasets
 
@@ -194,6 +233,8 @@ The [documentation](https://openvax.github.io/orfdata/) covers:
 
 | Task | Guide |
 | --- | --- |
+| Discover, inspect and download data | [Discover and download data](https://openvax.github.io/orfdata/downloads/) |
+| Use ORFs and viral references with varcode, isovar, topiary and vaxrank | [Downstream tools](https://openvax.github.io/orfdata/downstream/) |
 | Build a proteome: evidence levels, exclusions, contigs, viral proteins | [Build a reference proteome](https://openvax.github.io/orfdata/proteome/) |
 | ORF classes, evidence definitions, Varcode and Isovar integration | [ORFs and evidence](https://openvax.github.io/orfdata/reference/) |
 | Species and reference identity, mitochondrial genetic codes | [Species, references and genetic codes](https://openvax.github.io/orfdata/species/) |

{orfdata-0.35.0 → orfdata-0.36.0}/README.md

@@ -16,12 +16,37 @@ evidence.
 Use Python 3.11 or later. Install orfdata and one PyEnsembl annotation:
 
 ```sh
-python -m pip install 'orfdata[pyensembl,data,build-data]'
+python -m pip install 'orfdata[pyensembl,build-data]'
 pyensembl install --release 115 --species human
 # For the ncORF evidence presets and "all":
 orfdata setup --species human --catalogs all
 ```
 
+List every bundled, downloadable or buildable resource and what is installed:
+
+```sh
+orfdata download list
+orfdata download status
+```
+
+`orfdata setup --species human` also installs `gdc-viruses`, the default
+200-reference GDC viral screen for human transcriptome analysis;
+`--collections none` skips it. Viral support is part of the base install, and
+only the data is downloaded. To acquire only the viral references:
+
+```sh
+orfdata download viral --species human
+orfdata viral summary
+```
+
+Each viral collection keeps three layers apart — the viral genomes, one coding
+transcript per source CDS feature with a GTF placing it on them, and the
+proteins those transcripts encode. Because the three share identifiers, the set
+opens as a PyEnsembl genome, so varcode, isovar, topiary and vaxrank can
+annotate variants on viral contigs. `orfdata proteome --annotation DIR` writes
+the same three files for noncanonical ORFs. See
+[Downstream tools](https://openvax.github.io/orfdata/downstream/).
+
 ## Example
 
 ```python
@@ -108,6 +133,13 @@ amino acids, parent protein accession, sample, change IDs and source before
 `combined.with_derived_proteins(...)` adds them. These sequences are not
 inferred from variants or included in the reference proteome by default.
 
+For a named, checksummed sequence input with no genomic placement, use the
+general `ProteinSet`. `with_protein_set(...)` records a donor, transgene or
+other selected component; `with_viral(...)` uses the same input plus a sourced
+host association. ncORF sources with transcript coordinates and evidence stay
+as `ORFCatalog` records and produce the same `ProteinSequence` output when
+bound to a reference.
+
 ```sh
 orfdata proteome --species human --release 115 --nc-orf-sources all --viral all --output human-all-proteins.fasta
 ```
@@ -127,10 +159,14 @@ packaged bases and coding paths. The functional loaders remain available.
 When annotating an existing alignment, `viral_orfs(reference_dna,
 contig="HPV16")` checks its actual reference sequence before applying the same
 models.
-The first [coding reference](https://openvax.github.io/orfdata/viral/) contains
-the nine RefSeq NC_001526.4 HPV16 CDSs, with native and GDC placements;
-the host association is distinct from the virus's own taxon. Broader
-imports remain in [the viral roadmap](https://github.com/openvax/orfdata/issues/91).
+The curated [coding reference](https://openvax.github.io/orfdata/viral/)
+contains the nine RefSeq NC_001526.4 HPV16 CDSs, with native and GDC
+placements; the host association is distinct from the virus's own taxon. The
+separate `gdc-viruses` source bundle contains all 200 GDC source genomes and
+their annotated CDS/protein FASTAs for human analysis. It does not assert a
+human source host for every virus; source-host metadata remains separate.
+Promoting further source records into the curated
+`ViralReference` API remains in the [viral roadmap](https://github.com/openvax/orfdata/issues/91).
 
 ## Measurement datasets
 
@@ -162,6 +198,8 @@ The [documentation](https://openvax.github.io/orfdata/) covers:
 
 | Task | Guide |
 | --- | --- |
+| Discover, inspect and download data | [Discover and download data](https://openvax.github.io/orfdata/downloads/) |
+| Use ORFs and viral references with varcode, isovar, topiary and vaxrank | [Downstream tools](https://openvax.github.io/orfdata/downstream/) |
 | Build a proteome: evidence levels, exclusions, contigs, viral proteins | [Build a reference proteome](https://openvax.github.io/orfdata/proteome/) |
 | ORF classes, evidence definitions, Varcode and Isovar integration | [ORFs and evidence](https://openvax.github.io/orfdata/reference/) |
 | Species and reference identity, mitochondrial genetic codes | [Species, references and genetic codes](https://openvax.github.io/orfdata/species/) |

orfdata-0.36.0/docs/downloads.md

@@ -0,0 +1,257 @@
+# Discover and download data
+
+ORFdata has one common entry point for data acquisition:
+
+```sh
+orfdata download list
+orfdata download status
+orfdata download viral --species human
+orfdata download catalog NAME
+orfdata download dataset NAME
+orfdata download reference DATASET
+orfdata download genome NAME
+orfdata download chains --species human
+```
+
+The operation comes first so the same discovery and installation model applies
+to every resource kind. `orfdata download viral` is therefore the primary
+command. Focused command groups such as `orfdata viral coverage` and
+`orfdata viral summary` remain useful for domain-specific inspection.
+
+`list` shows every bundled, downloadable or locally buildable primary resource
+and its current state, including native protein search references declared by
+quantitative datasets. `status` shows resources already usable from local
+files. Filter either command with `--kind` or `--species`; use `--installed` or
+`--missing` with `list`, `--json` for programs, and `--verify` to hash installed
+files. Discovery and status commands never acquire or repair data.
+
+```sh
+orfdata download list --kind viral --species human
+orfdata download list --missing
+orfdata download status --json
+```
+
+Catalogue and dataset releases that cannot be redistributed are built from
+their pinned sources when selected through `orfdata download`. The listing's
+`ACQUIRE` column says `download` or `build`. Existing focused commands such as
+`orfdata catalogs download`, `orfdata datasets build`, `orfdata genomes
+install`, and `orfdata chains download` remain available.
+
+The same inventory is available from Python:
+
+```python
+from orfdata import DataStatus, list_data
+
+all_resources: tuple[DataStatus, ...] = list_data()
+human_missing: tuple[DataStatus, ...] = list_data(species="human", installed=False)
+installed: tuple[DataStatus, ...] = list_data(installed=True)
+machine_record: dict[str, object] = all_resources[0].to_dict()
+```
+
+Each `DataStatus` reports its kind, stable name and version, species,
+nonexclusive analysis contexts, local status, acquisition method and equivalent
+command, and resolved cache path. Status values follow DataCache: `available`,
+`missing`, `corrupt`, or `inaccessible`. An unknown kind, species or context is
+an error rather than an empty listing, so a typo never reads as "nothing is
+available".
+
+## Viral collections and contexts
+
+The Python API uses the same collection object for discovery, inventory reads,
+source summaries, downloads and offline opens. The target species comes from the
+collection; pass `species` only when you want an explicit consistency check.
+
+```python
+from pandas import DataFrame
+
+from orfdata import ViralCollection, viral_collection, viral_collections
+
+human_collections: tuple[ViralCollection, ...] = viral_collections(species="human")
+collection: ViralCollection = viral_collection("gdc-viruses")
+assert viral_collection(collection.panel) is collection
+
+inventory: DataFrame = collection.inventory()
+genbank: DataFrame = collection.source_summary()       # primary source
+pave: DataFrame = collection.source_summary("pave")    # another declared source
+```
+
+Names, panel names and aliases resolve through `viral_collection()` everywhere.
+An unknown species, collection or source is an error rather than an empty
+result. A second species is another `ViralCollection` declaration with its own
+checked inventory and source summaries; download code does not branch on the species.
+
+The published viral collection currently has these properties:
+
+| Property | Value | Meaning |
+| --- | --- | --- |
+| Species | `human` | Current target analysis species, NCBI taxon 9606 |
+| Collection | `gdc-viruses` | Default 200-reference GDC viral screen for human analysis |
+| Contexts | `cancer-transcriptome-screening` (`cancer`) | GDC human cancer-transcriptome screening context |
+
+Run the default collection with:
+
+```sh
+orfdata download viral --species human
+```
+
+Viral support is part of a regular `pip install orfdata`; what needs acquiring
+is the data. `orfdata setup --species human` installs this collection along
+with the human catalogues and chains; `--collections none` sets up catalogues
+and chains alone.
+
+`--collection gdc-viruses` is the default and is available for explicit
+configuration in scripts that will later choose among several collections. The
+current collection is for human analysis. A future collection may target another
+species. Target species does not assert a human source host for every virus.
+Context is metadata rather than a sequence identity or a mutually exclusive
+selector. CMV and EBV, for example, may be requested in both cancer and
+autoimmune workflows. A future collection may carry several context tags, and
+the same exact accession/version can belong to several collections without
+becoming several biological references.
+
+Pass `--context` to materialize a derived FASTA set from the shared source
+collection. Repeated contexts select their union; `--context-match all` selects
+their intersection. It reuses source records already downloaded for the
+collection, so overlapping contexts do not create conflicting copies of a
+reference.
+
+```sh
+orfdata viral contexts
+orfdata download viral --context cancer  # all 200 GDC screening references
+# Once another listed context is published, select its union or intersection:
+orfdata download viral --context cancer-transcriptome-screening --context <another-listed-context>
+orfdata download viral --context cancer-transcriptome-screening --context <another-listed-context> --context-match all
+```
+
+The target species is an admission and organization choice for the reference
+set. It does not replace a source record's viral taxon or assert that the source
+declared a host. Unknown source-host metadata remains unknown.
+
+The four facts stay separate in the downloaded tables and FASTA headers:
+the versioned viral accession and CDS feature identify the source product;
+collection and context record where it was selected; the target species records
+the analysis it supports; and a source-host taxon is populated only when the
+source establishes one. This keeps the same reference idempotent when it is
+obtained through several collections or exporter paths.
+
+The published GDC source inventory is named `GRCh38.d1.vd1`; `gdc-viruses` is
+the ORFdata collection name for its 200 references for human analysis.
+
+Use the full GDC collection for broad screening. For its three cancer
+selections, list named subsets with `orfdata viral subsets`:
+
+```sh
+orfdata download viral --subset cancer-integrated-driver
+orfdata download viral --subset cancer-causing
+orfdata download viral --subset cancer-associated
+```
+
+`cancer-causing` is the 20-reference IARC Group 1 selection.
+`cancer-associated` is its 37-reference Group 1, 2A and 2B superset.
+`cancer-integrated-driver` contains the 12 Group 1 HPV types, HBV, HTLV-1 and
+MCV. The remaining 163 GDC entries are 162 HPV references outside those IARC
+tiers plus SV40; their absence from the cancer-associated set does not establish
+that they are biologically irrelevant. IARC Group 3 means an agent is not
+classifiable as to carcinogenicity in humans; it is not a safety finding. Future
+respiratory and autoimmune contexts can overlap the GDC screening context.
+
+## Three layers, named separately
+
+A downloaded collection keeps the molecular layers apart, and ORFdata uses the
+same three words everywhere for them:
+
+| Layer | Molecule | Files | What it is |
+| --- | --- | --- | --- |
+| `genome` | DNA | `genomes.fasta`, `genomes.tsv` | Viral genomic DNA: the same molecules a DNA reference such as `GRCh38.d1.vd1` carries as extra contigs |
+| `transcriptome` | RNA | `transcripts.fasta`, `transcripts.tsv`, `annotation.gtf` | One coding transcript per source-annotated CDS feature, spliced, placed on the genome |
+| `proteome` | protein | `proteins.fasta` | The source translation of each coding transcript |
+
+A viral genome is not a transcript and a coding transcript is not a protein, so
+nothing in the cache mixes them: each layer has its own FASTA, its own
+identifiers and its own record count. Every transcript also records which
+source annotated it, and which end, if either, that source marks as running
+past the record.
+
+```python
+from orfdata import open_viral_references
+
+references = open_viral_references(species="human")
+for layer in references.layers:
+    print(layer.name, layer.molecule, layer.records, layer.fasta)
+transcriptome = references.layer("transcriptome")
+print(transcriptome.annotation)  # annotation.gtf
+```
+
+Summarize an installed collection, or one selection of it, without writing
+anything:
+
+```sh
+orfdata viral summary
+orfdata viral summary --subset cancer-causing --json
+orfdata viral completeness --incomplete
+```
+
+Because the GTF, transcript FASTA and protein FASTA share identifiers, the set
+opens as a PyEnsembl genome for downstream variant tools — see
+[Downstream tools](downstream.md).
+
+## Cache output
+
+Downloads have a canonical, versioned path under `ORFDATA_CACHE`,
+`OPENVAX_DATA_CACHE`, or the normal OpenVax cache. This is why the command does
+not require `--output`: an arbitrary export path would also need to define
+resource identity, update and verification behavior. Use `--cache-root` to put
+the managed cache elsewhere. The command prints the installed paths; add
+`--json` for structured output.
+
+## Python acquisition convention
+
+ORFdata follows the OpenVax split between explicit acquisition and offline
+lookup:
+
+```python
+from orfdata import ViralReferenceFiles, download_viral_references, open_viral_references
+
+download_viral_references(collection="gdc-viruses")
+references: ViralReferenceFiles = open_viral_references(collection="gdc-viruses")
+```
+
+The convenience form is explicit and opt-in:
+
+```python
+references = open_viral_references(collection="gdc-viruses", download_if_missing=True)
+```
+
+Context selection is a separate, type-stable API. It returns paths and counts
+for a derived reference set; it uses the installed collection above and fetches it only
+when requested:
+
+```python
+from orfdata import select_viral_references
+
+cancer = select_viral_references(
+    subsets=["cancer-causing"],
+    species="human",
+    download_if_missing=True,
+)
+
+integrated_drivers = select_viral_references(
+    subsets=["cancer-integrated-driver"],
+    download_if_missing=True,
+)
+```
+
+The default is `download_if_missing=False`, so opening or inspecting a resource
+does not unexpectedly use the network. The name matches the lower-level
+PyEnsembl/DataCache convention; high-level PyEnsembl and MHCflurry workflows
+also expose explicit download or fetch commands.
+
+Across ORFdata download APIs, repeated acquisition reuses a valid installation
+and `force=True` refreshes or repairs it, status objects provide `to_dict()`,
+human CLI output is the default, and `--json` provides machine-readable output.
+
+A derived selection is local: `select_viral_references(force=True)` rebuilds it from
+the cached collection without downloading, and a damaged selection is rebuilt rather
+than reported, since nothing in it is unrecoverable. Adding
+`download_if_missing=True` to that call lets it repair the source collection too,
+which does download.

orfdata-0.36.0/docs/downstream.md

@@ -0,0 +1,168 @@
+# Use ORFdata with varcode, isovar, topiary and vaxrank
+
+Those tools read a [PyEnsembl](https://github.com/openvax/pyensembl) genome: a
+GTF of transcript models, plus transcript and protein FASTAs whose identifiers
+match the GTF. ORFdata writes exactly that for the two things a reference
+annotation leaves out — noncanonical ORFs and viral references — so a variant
+in either gets a coding consequence from the same tools that annotate reference
+transcripts.
+
+## Three layers, three jobs
+
+ORFdata keeps the molecular layers apart and names them the same way in every
+command and API:
+
+| Layer | Molecule | What it answers | Who reads it |
+| --- | --- | --- | --- |
+| `genome` | DNA | Where a read or a variant call sits | Aligners; the contigs a VCF names |
+| `transcriptome` | RNA | What a variant does to a reading frame | varcode, isovar |
+| `proteome` | protein | Which peptide a patient could present | topiary, vaxrank, MS search |
+
+A DNA reference such as GDC's `GRCh38.d1.vd1` is the genome layer: human
+chromosomes plus the viral sequences as extra contigs. It carries no transcript
+models and no proteins, which is what the other two layers supply.
+
+## Viral transcripts in a tumour sample
+
+Install the collection, or one cancer subset of it, and open it as a genome:
+
+```sh
+orfdata download viral --species human --subset cancer-causing
+orfdata viral summary --subset cancer-causing
+```
+
+```python
+from varcode import Variant
+
+from orfdata import select_viral_references, viral_genome
+
+references = select_viral_references(
+    subsets=["cancer-causing"], species="human", download_if_missing=True
+)
+genome = viral_genome(references)  # A PyEnsembl genome over the viral contigs.
+
+variant = Variant(contig="AY446894.2", start=1500, ref="C", alt="T", genome=genome)
+print(variant.effects().top_priority_effect().short_description)
+```
+
+`viral_genome()` indexes the set's `annotation.gtf`, `transcripts.fasta` and
+`proteins.fasta` into a PyEnsembl database kept beside the cache, never inside
+the checksummed reference directory. The same object goes to isovar with the
+tumour RNA alignment, and to topiary or vaxrank with an MHC predictor, wherever
+those take a genome.
+
+Four things are worth knowing before the first run:
+
+- **Contig names are the source `accession.version`**, such as `AY446894.2`,
+  because that is the record the sequences and the annotation come from.
+  `genomes.tsv` keeps the inventory's `source_genbank` and `source_refseq`
+  spellings beside it, so you can map the names your alignment used.
+- **A non-standard genetic code is carried**, as `transl_table` on the
+  transcript and CDS rows, so PyEnsembl translates with the code the source
+  record declares.
+- **A partial or frameshifted source CDS gets no start or stop codon rows**, so
+  effect predictors will not translate it. `annotation_complete` says which
+  transcripts are whole and `partial` says which end the source marks as
+  running past the record: 78 of 1,703, mostly HPV E4 and E5 ORFs that are
+  really expressed from spliced transcripts, plus HIV-1's `pol`, which is a
+  Gag-Pol frameshift fusion. HPV16's E6, E7, E1, E2, L1 and L2 are complete,
+  and HBV's polymerase is joined back together across the circular origin, so
+  variants on either side of it get a coding consequence.
+- **A coding transcript here is the CDS span**, not a UTR-containing mRNA. The
+  source records annotate 1,678 CDS features but only 39 `mRNA` features, so
+  the transcriptome layer is built from the CDS features and says so rather
+  than inventing transcript bounds.
+- **Three source records annotate no CDS at all** — HPV2, HPV8 and HPV51 — so
+  their coding features come from PaVE instead, on the identical sequence, and
+  every transcript records which source annotated it. Use `--no-annotation-fill`
+  to keep strictly to the GenBank submissions; `orfdata viral summary` then
+  names the genomes that are left without transcripts. For the opposite
+  trade — one source's uniform, spliced gene models everywhere it publishes
+  the same molecule, in place of each submission's own naming — use
+  `--alternate-annotations`, which leaves the genome FASTA byte-identical and
+  adds 235 coding transcripts across the GDC collection. `orfdata viral
+  completeness` shows what either choice leaves on the table.
+- A transcript's `initiation` says how the source starts it (`atg`,
+  `alternative`, `declared`, `non_canonical`, `partial`, `unknown`) and
+  `recoding` names any frameshift the source claims. Effect predictors handle
+  a non-canonical start correctly, because the protein a source publishes is
+  the one in `proteins.fasta`; they do not handle a variant at the base a
+  ribosomal-frameshift join reads twice, which is two positions in HTLV-1.
+
+## Noncanonical ORFs
+
+Build a proteome as usual and ask for the matching annotation:
+
+```sh
+orfdata proteome --species human --release 115 --nc-orf-sources ms \
+  --output proteins.fasta --annotation ncorfs/
+```
+
+That writes `ncorfs/annotation.gtf`, `ncorfs/transcripts.fasta` and
+`ncorfs/proteins.fasta`: one coding transcript per ncORF, placed on its host
+gene, with the spliced coding sequence and the translation. In Python the same
+is `proteome.write_annotation("ncorfs")`.
+
+```python
+from pyensembl import Genome
+
+genome = Genome(
+    reference_name="GRCh38",
+    annotation_name="orfdata-ncorfs",
+    annotation_version="1",
+    gtf_path_or_url="ncorfs/annotation.gtf",
+    transcript_fasta_paths_or_urls=["ncorfs/transcripts.fasta"],
+    protein_fasta_paths_or_urls=["ncorfs/proteins.fasta"],
+    cache_directory_path="ncorfs/index",
+)
+genome.index()
+```
+
+A variant inside an ncORF now has a protein consequence — a stop gained in a
+lncRNA ORF reads as `p.K2*` rather than disappearing into "non-coding
+transcript".
+
+To keep the reference transcripts as well, concatenate the files instead of
+replacing them. PyEnsembl takes several FASTAs, and the ncORF GTF writes no
+`gene` rows precisely so it can be appended to an annotation that has them:
+
+```sh
+cat Homo_sapiens.GRCh38.115.gtf ncorfs/annotation.gtf > combined.gtf
+```
+
+```python
+genome = Genome(
+    reference_name="GRCh38", annotation_name="ensembl-plus-ncorfs", annotation_version="115",
+    gtf_path_or_url="combined.gtf",
+    transcript_fasta_paths_or_urls=["Homo_sapiens.GRCh38.cdna.all.fa", "ncorfs/transcripts.fasta"],
+    protein_fasta_paths_or_urls=["Homo_sapiens.GRCh38.pep.all.fa", "ncorfs/proteins.fasta"],
+    cache_directory_path="combined/index",
+)
+```
+
+For the models alone, without building a proteome, the query command writes the
+same GTF rows for any selection:
+
+```sh
+orfdata --evidence ms --format gtf > ncorfs.gtf
+orfdata --gene LARP4B --format gtf
+```
+
+Two deliberate choices in that file:
+
+- ncORF transcripts are marked `transcript_biotype "protein_coding"`, because
+  each has a complete CDS with start and stop codons and effect predictors only
+  translate transcripts marked that way. The source's own category stays in
+  `orf_biotype`, and the host transcript in `host_transcript_id`.
+- The transcript is the ORF's coding path, so its identifier is the catalogue's
+  `orf_id`. The protein is `<orf_id>.p`. Nothing is renamed to look like an
+  Ensembl accession.
+
+## What this does not claim
+
+An ORF model here is a catalogue ORF placed on an assembly, with the evidence
+the source reported; a predicted coding consequence is not evidence that a
+stable protein exists or that a peptide is presented. Viral collection and
+context membership record where a reference was selected, not that a virus
+drives a particular tumour. The layers stay separate so each of those questions
+can be asked of the right one.

{orfdata-0.35.0 → orfdata-0.36.0}/docs/index.md

@@ -12,13 +12,23 @@ is a recorded host of the virus.
 Use Python 3.11 or later. Install orfdata and one PyEnsembl annotation:
 
 ```sh
-python -m pip install 'orfdata[pyensembl,data,build-data]'
+python -m pip install 'orfdata[pyensembl,build-data]'
 pyensembl install --release 115 --species human
 orfdata setup --species human --catalogs all  # For ncORF presets and "all"
 ```
 
-The second command downloads and indexes Ensembl 115 for human GRCh38. orfdata
-never downloads or indexes annotation data itself; it reports what is missing.
+Viral support is part of the base install; only the data is downloaded. Get
+the human GDC viral references — genomes, coding transcripts and proteins, as
+three named layers — with `orfdata download viral --species human`. Use
+`orfdata download list` to see every known resource and `orfdata download
+status` to see what is local. To annotate variants in viral transcripts or
+noncanonical ORFs with varcode, isovar, topiary or vaxrank, see
+[Downstream tools](downstream.md).
+
+The second command downloads and indexes Ensembl 115 for human GRCh38. Opening
+ORFdata resources never downloads them implicitly; use the explicit commands
+under `orfdata download` or the corresponding PyEnsembl command when data is
+missing.
 
 ## Build a human proteome