taxdumpy 1.2.1__tar.gz → 1.3.0__tar.gz

{taxdumpy-1.2.1 → taxdumpy-1.3.0}/.gitignore

@@ -206,6 +206,12 @@ marimo/_static/
 marimo/_lsp/
 __marimo__/
 
+# Taxdump caches generated beside the .dmp files when tests or tools run
+taxdump.pickle
+taxdump_fast.pickle
+taxdump.sqlite
+taxdump.sqlite.*.tmp
+
 # More
 .DS_Store
 CLAUDE.md

taxdumpy-1.3.0/AGENTS.md

@@ -0,0 +1,287 @@
+# Repository Guidance
+
+Instructions and verified project notes for agents working in this repository.
+
+## Rules
+
+- Do not add "Co-Authored-By" messages in git commits
+
+## Project Overview
+
+**Taxdumpy** is a high-performance Python toolkit for parsing NCBI Taxonomy databases with lineage resolution and fuzzy search capabilities. Designed for metagenomics, phylogenetics, and bioinformatics applications.
+
+- **Version source**: `src/taxdumpy/__init__.py` (Hatch reads the version from this file)
+- **Python**: 3.10+
+- **License**: MIT
+- **Repository visibility**: GitHub repository `omegahh/taxdumpy` is private.
+  Codecov coverage uploads require a Codecov account and repository token; the
+  current CI upload is non-blocking if Codecov rejects or cannot process it.
+
+## Development Commands
+
+### Installation and Setup
+```bash
+# Install in development mode
+python -m pip install -e .
+
+# Install with dependencies
+python -m pip install -e '.[dev]'
+```
+
+### Running the CLI
+```bash
+# Cache taxonomy database (requires NCBI taxdump files)
+taxdumpy cache -d /path/to/taxdump
+
+# Search for organisms with fuzzy matching
+taxdumpy search --fast "Escherichia coli"
+
+# Get lineage for a specific taxid
+taxdumpy lineage 511145
+```
+
+`search --fast` and `lineage --fast` need a filtered pickle cache created with
+`cache -f FILE`. Without `--fast`, both commands use SQLite to avoid loading the
+full pickle for a single query. Python batch callers should create one `TaxDb`
+instance and reuse it for all queries.
+
+### Testing and Quality
+```bash
+# Run all tests with coverage (requires 80% minimum)
+pytest
+
+# Run specific test categories
+pytest -m "not slow and not integration"  # Fast tests only
+pytest -m "slow"                          # Slow tests only
+pytest -m "integration"                   # Integration tests only
+pytest -m "unit or not (slow or integration)"  # Unit and unmarked non-slow tests
+
+# Run tests with coverage report
+pytest --cov=taxdumpy --cov-report=html --cov-report=term-missing
+
+# Run a focused test file without the repository-wide coverage threshold
+pytest tests/test_taxdb.py --no-cov
+
+# Format code and sort imports with ruff
+ruff format src/ tests/  # Format code
+ruff check src/ tests/ --fix  # Fix import sorting
+
+# Check formatting and imports in CI mode
+ruff format --check src/ tests/
+ruff check src/ tests/  # Only checks import sorting (I rules)
+```
+
+**Note**: Ruff is configured with `line-length = 120`, and only handles formatting and import sorting (I rules). All other linting is disabled.
+
+**Note**: pytest config lives in `pytest.ini` under a `[pytest]` section. It must
+not be `[tool:pytest]` — that is the `setup.cfg` spelling, and pytest silently
+ignores the whole file, taking `--strict-markers` and the 80% coverage gate with
+it. Running `pytest` requires `pytest-cov` to be installed, since `addopts`
+passes `--cov` unconditionally.
+
+Test fixtures copy the `.dmp` files into temporary directories, and backend
+caches are written there. Pytest also writes coverage reports in the repository
+root; those reports and cache filenames are gitignored.
+
+### Build and Release
+```bash
+# Build an sdist and wheel (requires the `build` package)
+python -m build
+
+# A pushed v* tag triggers GitHub Release creation; it does not publish to PyPI.
+# Build and upload separately with Twine when a PyPI release is intended.
+# python -m twine upload dist/*
+```
+
+`scripts/release.py` has remote and repository side effects: it requires a clean
+worktree, updates the version, commits it, pushes `origin main` and a version
+tag, removes the existing `dist/` directory, and builds the package. Its
+`--upload` option publishes to PyPI. Read the script and use it only when those
+exact actions are intended.
+
+## Architecture Overview
+
+### Project Structure
+```
+src/taxdumpy/
+├── __init__.py          # Package initialization and exports
+├── __main__.py          # CLI entry point
+├── ansi.py              # ANSI color formatting utilities
+├── basic.py             # Core data structures (Node) and exception hierarchy
+├── ranks.py             # Shared canonical rank names, levels and codes
+├── database.py          # Abstract base class (TaxonomyDatabase) + factory function
+├── cache.py             # Source signatures and atomic pickle replacement
+├── taxdb.py             # Dictionary-based in-memory backend with pickle caching
+├── taxsqlite.py         # SQLite-based persistent backend
+├── taxon.py             # Taxon entity with lineage resolution
+├── functions.py         # Rank utilities (upper_rank_id, get_canonical_ranks) + subtree traversal (descendants)
+└── cli.py               # CLI commands: cache, search, lineage
+
+tests/
+├── conftest.py          # Pytest fixtures
+├── data/                # Sample taxdump files for testing
+├── test_api_consistency.py
+├── test_basic.py
+├── test_cache.py
+├── test_cli.py
+├── test_critical_regressions.py  # Pins previously-shipped critical defects
+├── test_descendants.py           # Subtree traversal, parametrized over both backends
+├── test_error_handling.py
+├── test_fast_cache.py
+├── test_functions.py
+├── test_integration.py
+├── test_taxdb.py
+├── test_taxon.py
+└── test_taxsqlite.py
+```
+
+### Core Components
+
+#### Database Layer
+
+**TaxonomyDatabase** (`database.py`): Abstract base class defining the consistent API
+- Abstract methods: `get_node()`, `_rapid_fuzz()`, `__len__()`, `__repr__()`
+- Shared `search()` returns structured results; `fuzzy_search()` prints them
+- Shared methods: `_validate_taxid()` - validates and converts taxid to integer
+- Abstract cached properties: `all_names`, `name2taxid`, `delnodes`, `max_taxid_strlen`, `max_rank_strlen`
+- Context manager support for connection management
+- Formatting width properties return zero for an empty database
+
+**Important**: these five properties use the module's `abstract_cached_property`
+helper, not `@cached_property` over `@abstractmethod`. Stacking those two
+silently breaks enforcement — `cached_property` does not propagate
+`__isabstractmethod__`, so ABCMeta never sees the member and a subclass omitting
+it instantiates fine, returning `None`. Do not "simplify" this back.
+
+**TaxDb** (`taxdb.py`): Dictionary-based in-memory database
+- Fast mode (`fast=True`) loads `taxdump_fast.pickle`, built from the taxids in a
+  filter file, their ancestors, and their descendants
+- Deserializes a pickle once per initialization; reuse the instance for batches
+- `build_fast_cache(taxids)` validates selections and writes the filtered cache
+- New pickle caches include versioned source signatures; original three-part
+  pickles remain readable. Changes in available source sizes or mtimes trigger
+  a rebuild; caches remain usable when source files have been removed
+- Pickle writes use atomic replacement. `rebuild=True` forces a full rebuild;
+  a failed rebuild preserves the previous cache
+- Missing, stale or corrupt fast caches require an explicit filtered rebuild
+- Data structures: `_taxid2nodes` (dict), `_old2news` (merged taxids), `_delnodes` (set)
+
+**TaxSQLite** (`taxsqlite.py`): SQLite-based persistent database
+- Tables: `nodes` (taxid, parent, rank, division), `names` (taxid, name, name_class), `merged` (oldtid, newtid), `delnodes` (taxid)
+- All four tables are listed in `REQUIRED_TABLES`; `_check_tables` validates every one, so a cache built by an older version with a narrower schema is rebuilt rather than trusted
+- The database is self-contained: no `.dmp` file is read at query time
+- Builds go to a `mkstemp` temp path and are `os.replace`d into position, so an interrupted build cannot leave a schema-complete but empty database behind
+- Indexed for fast queries
+- Deleted-taxid checks use an indexed lookup rather than loading the whole set
+- New caches record source signatures in `source_files`; legacy caches use
+  source mtimes. `rebuild=True` forces a rebuild with atomic replacement
+- Memory-efficient for large datasets
+
+**Factory Function**: `create_database(taxdump_dir, backend="sqlite", ...)`.
+Supported backend names are `sqlite`/`sql` and `dict`/`memory`/`pickle`.
+
+#### Data Structures
+
+**Node** (`basic.py`): Dataclass with `__slots__` optimization
+- Fields: `taxid`, `parent`, `rank`, `name`, `equal`, `acronym`, `division`
+
+**Taxon** (`taxon.py`): Main entity for taxonomy operations
+- Properties: `lineage`, `taxid_lineage`, `rank_lineage`, `name_lineage`
+- Properties: `is_legacy`, `has_species_level`, `species_taxid`
+- Traces lineage from any taxid to root, handles merged taxids automatically
+
+#### Exception Hierarchy
+
+```
+TaxdumpyError (base)
+├── TaxDbError          # Database-related issues
+│   ├── TaxdumpFileError        # File issues (includes solutions)
+│   └── DatabaseCorruptionError
+├── TaxidError          # Invalid/unknown taxid (includes suggestions)
+├── TaxRankError        # Invalid taxonomic rank (includes valid ranks)
+├── TaxdumpyMemoryError
+└── ValidationError     # Input validation failures
+```
+
+#### Rank Functions (`functions.py`)
+
+- Rank tables live in `ranks.py` and are re-exported by `functions.py`
+- `RankName` defines 9 levels plus the `domain` alias of `superkingdom`;
+  `RANKNAMES` is derived via `get_args` (species to realm)
+- Realm uses the root-level `R` code to preserve existing integer encodings,
+  and `r__` in MPA output. Root itself is omitted from MPA output
+- `EXTENDED_RANKS`: 32 names including sub-ranks and aliases
+- `upper_rank_id()`: Get taxid at specific canonical rank (strict)
+- `get_rank_taxid()`: Safely get taxid for rank (returns None if not found)
+- `get_canonical_ranks()`: Get all canonical ranks in lineage as dict
+- `get_closest_rank()`: Get closest rank at or above target level
+- `get_rank_or_closest()`, `get_all_ranks()`, `find_rank_in_lineage()`
+- `get_rank_distance()`: Calculate distance between two canonical ranks
+- `group_taxids_by_rank()`: Group taxids by rank using their closest ancestor
+
+#### Subtree Traversal (`functions.py`)
+
+Lineage tracing walks *up*; these walk *down*.
+
+- `descendants(taxdb, taxid, max_depth=None)`: every taxid below `taxid`,
+  excluding `taxid` itself and excluding deleted/merged taxids. Breadth-first.
+  `max_depth=1` gives direct children only
+- `descendant_nodes()`: the same as `Node` objects (one `get_node` per result)
+- `build_children_index(taxdb)`: inverts child→parent in one pass, cached per
+  database in a `WeakKeyDictionary` so a closed database does not pin a
+  million-entry index. Traversal is iterative, so deep trees cannot exhaust the
+  stack
+
+These reach into `taxdb._taxid2nodes` and `taxsqlite._conn` for speed, so they
+dispatch on backend type and raise `TaxDbError` for any third backend. Moving
+them onto `TaxonomyDatabase` would remove that coupling.
+
+### Data Flow
+1. NCBI taxdump files (nodes.dmp, names.dmp, etc.) are parsed
+2. Data is cached as either pickle files (TaxDb) or SQLite database (TaxSQLite)
+3. Taxon objects provide lineage tracing and taxonomy operations
+4. CLI commands leverage either backend for search/lineage operations
+
+### CLI Commands
+
+| Command | Description | Key Options |
+|---------|-------------|-------------|
+| `cache` | Build SQLite and full pickle caches; with `-f`, build only a filtered fast pickle | `-d/--directory`, `-f/--filter-file` |
+| `search` | Fuzzy organism name search | `-d/--directory`, `--fast`, `--limit` |
+| `lineage` | Display taxonomic lineage (SQLite by default, filtered pickle with `--fast`) | `-d/--directory`, `--fast` |
+
+### Environment Variables
+- `TAXDB_PATH`: Default path to taxonomy database (defaults to `~/.taxonkit`)
+
+### Performance Features
+- Pickle caching for repeated Python queries; `fast=True` loads a filtered cache
+- SQLite backend for memory efficiency with large datasets
+- Rapidfuzz integration for fuzzy organism name matching
+- Fast cache mode for selected taxids and their related lineage/subtree
+
+### Database Requirements
+Building a new cache from source requires NCBI taxdump files:
+- `nodes.dmp`: Taxonomy tree structure
+- `names.dmp`: Scientific names
+- `merged.dmp`: Legacy taxid mappings
+- `delnodes.dmp`: Deleted nodes
+- `division.dmp`: Division information
+
+### Dependencies
+- **Core**: `rapidfuzz` (fuzzy matching), `tqdm` (progress bars)
+- **Dev**: `ruff`, `pytest`, `pytest-cov`, `pytest-mock`, `build`
+
+## Known Issues
+
+These issues are still present in the checked-in implementation. Recheck this
+section when the corresponding code or tests change.
+
+**Mutable nodes**: `TaxDb.get_node` returns stored Node objects while SQLite
+returns fresh ones. Mutating dict nodes can invalidate cached search and
+children indexes; treat returned nodes as read-only until an explicit mutation
+API or immutable node contract is introduced.
+
+**Importer differences**: malformed input handling differs between backends.
+SQLite construction still holds parsed names in memory, so its build uses more
+memory than steady-state queries. Node and insert rows are streamed, with
+indexes built after bulk insertion.

{taxdumpy-1.2.1 → taxdumpy-1.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: taxdumpy
-Version: 1.2.1
+Version: 1.3.0
 Summary: Python package for efficiently parsing NCBI's taxdump database
 Project-URL: Documentation, https://github.com/omegahh/taxdumpy
 Project-URL: Issues, https://github.com/omegahh/taxdumpy/issues
@@ -61,9 +61,10 @@ _A high-performance Python toolkit for parsing NCBI Taxonomy databases with line
 
 ## Features
 
-- **Fast Parsing**: Optimized loading of NCBI taxdump files with pickle caching for ~3x speedup
+- **Cached Loading**: Parse NCBI taxdump files once, then reuse pickle or SQLite caches
 - **Dual Backends**: In-memory (`TaxDb`) for speed or SQLite (`TaxSQLite`) for memory efficiency
 - **Lineage Resolution**: Complete lineage tracing with automatic handling of merged/deleted nodes
+- **Subtree Traversal**: Walk *down* the tree with `descendants` — every taxid below a given one
 - **Rank Codes**: Comparable `RC` class for efficient rank-based operations and grouping
 - **Fuzzy Search**: Rapid approximate name matching using `rapidfuzz`
 - **Batch Grouping**: Group taxids by rank level with `group_taxids_by_rank`
@@ -86,7 +87,7 @@ pip install -e .
 **With Development Dependencies:**
 
 ```bash
-pip install -e .[dev]
+pip install -e '.[dev]'
 ```
 
 ## Quick Start
@@ -112,7 +113,7 @@ print(human.rank_lineage)
 
 # Check taxonomic properties
 print(f"Rank: {human.rank}")           # species
-print(f"Division: {human.division}")   # Primates
+print(f"Division: {human.division}")   # PRI (division code, not full name)
 print(f"Is legacy: {human.is_legacy}") # False
 ```
 
@@ -121,23 +122,54 @@ print(f"Is legacy: {human.is_legacy}") # False
 ```python
 from taxdumpy import create_database, Taxon
 
-# Create database with preferred backend
-db = create_database(backend="sqlite", taxdump_dir="/path/to/taxdump")
+# taxdump_dir is the first positional argument
+db = create_database("/path/to/taxdump", backend="sqlite")
 # Or: backend="dict" for in-memory
 
 taxon = Taxon(9606, db)
 print(taxon.name)  # Homo sapiens
 ```
 
+### Subtree Traversal
+
+```python
+from taxdumpy import TaxDb, descendants
+
+taxdb = TaxDb("/path/to/taxdump")
+
+# Every taxid below genus Escherichia (transitively, not just children)
+descendants(taxdb, 561)
+# [562, 208962, 511145, ...]
+
+# Direct children only
+descendants(taxdb, 561, max_depth=1)
+
+# Deleted and merged taxids are excluded; the queried taxid itself is not
+# included. Combine with existing helpers to filter:
+species = [t for t in descendants(taxdb, 2) if taxdb.get_node(t).rank == "species"]
+```
+
+The first call builds a parent-to-children index in one pass over the database
+and caches it, so later queries cost time proportional to the subtree returned
+rather than to the database size.
+
 ### Fuzzy Search
 
 ```python
-# Search with typos and partial matches
-results = taxdb._rapid_fuzz("Escherichia coli", limit=5)
+# Print formatted, highlighted matches to the console
+taxdb.fuzzy_search("Escherichia coli", limit=5)
+
+# Or get the matches as data
+results = taxdb.search("Escherichia coli", limit=5)
 for match in results:
     print(f"{match['name']} (TaxID: {match['taxid']}, Score: {match['score']})")
 ```
 
+`search` returns dictionaries containing `name`, `taxid`, and a `score` from
+0 to 100. It preserves separate taxa sharing a name and reuses its candidate
+index across queries. `fuzzy_search` prints the results. Both methods propagate
+validation and database errors to callers.
+
 ### Rank Codes and Grouping
 
 ```python
@@ -169,6 +201,10 @@ print(human.mpa_repr)
 
 ## Command Line Interface
 
+By default, `search` and `lineage` use SQLite so individual queries do not load
+the full pickle into memory. `--fast` loads a smaller, filtered pickle instead;
+create that cache first with `cache -f FILE`.
+
 ### Check Version
 
 ```bash
@@ -178,13 +214,17 @@ taxdumpy --version
 ### Cache Database
 
 ```bash
-# Cache full NCBI taxonomy database
+# Rebuild both taxdump.sqlite and taxdump.pickle from source files
 taxdumpy cache -d /path/to/taxdump
 
-# Create fast cache with specific organisms
+# Build taxdump_fast.pickle from selected taxids (one per line), their
+# ancestors, and their descendants; blank lines and # comments are allowed
 taxdumpy cache -d /path/to/taxdump -f important_taxids.txt
 ```
 
+The filtered command loads the full `TaxDb` to build the subset. It does not
+rebuild SQLite. Queries outside the subset require the full database.
+
 ### Search Operations
 
 ```bash
@@ -250,7 +290,7 @@ export TAXDB_PATH=~/.taxonkit
 from taxdumpy import TaxSQLite, Taxon
 
 # Use SQLite for memory-efficient storage
-db = TaxSQLite("/path/to/taxdump")  # Creates taxonomy.db in that directory
+db = TaxSQLite("/path/to/taxdump")  # Creates taxdump.sqlite in that directory
 
 # Same API as TaxDb
 taxon = Taxon(9606, db)
@@ -264,11 +304,14 @@ with TaxSQLite("/path/to/taxdump") as db:
 
 ### Batch Processing
 
+Load the full pickle once and reuse the same instance for all Python queries.
+Its startup cost is paid once per instance, rather than once per taxid.
+
 ```python
 from taxdumpy import TaxDb, Taxon
 
-# Reuse database instance for efficiency
-taxdb = TaxDb("/path/to/taxdump", fast=True)
+# Load all taxa for arbitrary batch queries
+taxdb = TaxDb("/path/to/taxdump")
 
 taxids = [9606, 511145, 7227, 4932]  # Human, E.coli, Fly, Yeast
 for taxid in taxids:
@@ -276,6 +319,43 @@ for taxid in taxids:
     print(f"{taxon.name}: {' > '.join(taxon.name_lineage[:3])}")
 ```
 
+For a known subset, build a filtered cache using the full database, then load
+that smaller cache in subsequent processes:
+
+```python
+count = taxdb.build_fast_cache([9606, 511145, 7227, 4932])
+print(f"Cached {count} nodes, including ancestors and descendants")
+
+subset = TaxDb("/path/to/taxdump", fast=True)
+human = Taxon(9606, subset)
+```
+
+### Cache Freshness and Rebuilding
+
+New caches record source file sizes and modification times. Opening a stale
+full pickle or SQLite cache rebuilds it from the taxdump files. Missing, stale,
+or unreadable filtered caches raise an error; recreate them with `cache -f FILE`
+or `build_fast_cache(taxids)` so the selection is explicit.
+
+Original three-part pickles remain readable, and legacy caches use source
+modification times to detect changes. Valid caches remain usable after the
+source `.dmp` files are removed. Rebuilding requires the source files again.
+
+To force a rebuild from Python:
+
+```python
+from taxdumpy import TaxDb, TaxSQLite
+
+taxdb = TaxDb("/path/to/taxdump", rebuild=True)
+with TaxSQLite("/path/to/taxdump", rebuild=True) as db:
+    print(len(db))
+```
+
+Each cache is replaced atomically only after its new contents are written
+successfully. A failed rebuild preserves the previous file. `rebuild=True`
+cannot be combined with `fast=True`; rebuild filtered caches through the full
+database's `build_fast_cache` method instead.
+
 ### Handling Merged/Legacy TaxIDs
 
 ```python
@@ -293,7 +373,7 @@ print(taxon.node.taxid) # Current/resolved taxid
 ### Exception Handling
 
 ```python
-from taxdumpy import TaxDb, Taxon, TaxidError, TaxRankError, TaxDbError
+from taxdumpy import TaxDb, Taxon, TaxidError, TaxRankError, upper_rank_id
 
 taxdb = TaxDb("/path/to/taxdump")
 
@@ -304,7 +384,8 @@ except TaxidError as e:
     # Includes suggestions for similar taxids
 
 try:
-    rank_id = upper_rank_id(9606, "invalid_rank", taxdb)
+    # Signature is (tax, taxdb, rank) — a Taxon, not a bare taxid
+    rank_id = upper_rank_id(Taxon(9606, taxdb), taxdb, "invalid_rank")
 except TaxRankError as e:
     print(f"Invalid rank: {e}")
     # Includes list of valid ranks
@@ -317,32 +398,37 @@ except TaxRankError as e:
 **TaxDb**: In-memory dictionary-based database with pickle caching
 
 ```python
-TaxDb(taxdump_dir: str, fast: bool = False)
+TaxDb(taxdump_dir: Path | str, fast: bool = False, verbose: bool = False, *, rebuild: bool = False)
 ```
 
 **TaxSQLite**: SQLite-based persistent database
 
 ```python
-TaxSQLite(taxdump_dir: str)
+TaxSQLite(taxdump_dir: Path | str, verbose: bool = False, *, rebuild: bool = False)
 ```
 
 **Taxon**: Taxonomic unit with lineage resolution
 
 ```python
-Taxon(taxid: int, taxdb: TaxDb | TaxSQLite)
+Taxon(taxid: int | str, taxdb: TaxDb | TaxSQLite)
 
 # Properties
-.name: str               # Scientific name
-.rank: str               # Taxonomic rank
-.division: str           # NCBI division
-.parent: int             # Parent taxid
-.lineage: list[Node]     # Complete lineage as Node objects
-.name_lineage: list[str] # Names from taxid to root
-.rank_lineage: list[str] # Ranks from taxid to root
+.name: str                # Scientific name
+.rank: str                # Taxonomic rank
+.division: str            # NCBI division
+.parent: int              # Parent taxid
+.acronym: str | None      # Acronym, if any
+.node: Node               # Underlying node (resolved, if merged)
+.rc: RC                   # Rank code for this taxon
+.lineage: list[Node]      # Complete lineage as Node objects
+.name_lineage: list[str]  # Names from taxid to root
+.rank_lineage: list[str]  # Ranks from taxid to root
 .taxid_lineage: list[int] # Taxids from taxid to root
-.is_legacy: bool         # True if taxid was merged
-.has_species_level: bool # True if lineage contains species rank
-.species_taxid: int | None # Taxid at species level
+.rankcode_lineage: list[RC]  # Rank codes from taxid to root
+.mpa_repr: str            # MetaPhlAn-style lineage string
+.is_legacy: bool          # True if taxid was merged
+.has_species_level: bool  # True if lineage contains species rank
+.species_taxid: int | None # Taxid at species level, or None if absent (previously 0)
 ```
 
 **Factory Function**
@@ -351,21 +437,52 @@ Taxon(taxid: int, taxdb: TaxDb | TaxSQLite)
 from taxdumpy import create_database
 
 # backend: "sqlite", "sql", "dict", "memory", or "pickle"
-db = create_database(backend="sqlite", taxdump_dir="/path/to/taxdump")
+db = create_database("/path/to/taxdump", backend="sqlite")
 ```
 
 ### Rank Functions
 
 ```python
 from taxdumpy import (
-    upper_rank_id,      # Get taxid at rank (raises TaxRankError if not found)
-    get_rank_taxid,     # Get taxid at rank (returns None if not found)
+    upper_rank_id,       # Get taxid at rank (raises TaxRankError if not found)
+    get_rank_taxid,      # Get taxid at rank (returns None if not found)
     get_canonical_ranks, # Get dict of all canonical ranks in lineage
-    get_closest_rank,   # Get closest rank at or above target level
-    get_rank_distance,  # Calculate distance between two ranks
-    RANKNAMES,          # 8 canonical ranks: species → realm
-    EXTENDED_RANKS,     # 30+ ranks including sub-ranks
+    get_closest_rank,    # Get closest rank at or above target level
+    get_rank_or_closest, # Same, returning the taxid alone
+    get_all_ranks,       # Every rank in the lineage as a dict
+    find_rank_in_lineage,# First of several candidate ranks present
+    get_rank_distance,   # Calculate distance between two ranks
+    group_taxids_by_rank,# Group a list of taxids at a given rank
 )
+
+# The rank tables live in taxdumpy.functions, not the package root:
+from taxdumpy.functions import RANKNAMES, EXTENDED_RANKS
+# RANKNAMES:      9 canonical levels, species → realm (domain/superkingdom are aliases)
+# EXTENDED_RANKS: 32 names including sub-ranks and aliases
+```
+
+`domain` and `superkingdom` resolve to the same rank level. Realm uses `r__` in
+MPA output and the root-level `R` rank code, preserving existing integer `RC`
+values. Grouping by `root` always selects taxid 1, including viral lineages.
+
+`upper_rank_id` requires the requested higher rank or its subrank to exist in
+the lineage; otherwise it raises `TaxRankError`. Use `get_closest_rank` or
+`get_rank_or_closest` when fallback to a higher canonical rank is desired.
+
+### Subtree Traversal
+
+```python
+from taxdumpy import descendants, descendant_nodes, build_children_index
+
+descendants(taxdb, taxid, max_depth=None) -> list[int]
+# Every taxid below `taxid`. Excludes `taxid` itself, and excludes deleted
+# and merged taxids. max_depth=1 gives direct children only. Breadth-first.
+
+descendant_nodes(taxdb, taxid, max_depth=None) -> list[Node]
+# Same, as Node objects. Costs one get_node per result.
+
+build_children_index(taxdb) -> dict[int, list[int]]
+# The parent-to-children index the above build and cache. Read-only.
 ```
 
 ### Exceptions
@@ -384,11 +501,17 @@ from taxdumpy import (
 
 ## Performance Tips
 
-- **Use Fast Mode**: `TaxDb(path, fast=True)` provides ~3x speedup with pre-cached data
-- **Reuse Instances**: Create one `TaxDb` instance and reuse for multiple operations
+- **Single CLI Queries**: Use the default SQLite backend to avoid loading the full pickle
+- **Python Batches**: Load one `TaxDb` instance and reuse it for multiple operations
+- **Filtered Data**: Use `TaxDb(path, fast=True)` after building a subset cache;
+  startup cost and memory use depend on the subset size
 - **Environment Variables**: Set `TAXDB_PATH` to avoid repeating database paths
 - **Choose Backend**: Use `TaxSQLite` for large datasets with limited memory
 - **Batch Operations**: Process multiple TaxIDs in batches rather than individual calls
+- **Subtree Queries**: Reuse one database instance so `descendants` shares its
+  cached children index. Building it costs one pass over the database
+  (~0.4 s for 1M nodes); every query afterwards is proportional to the subtree,
+  and small subtrees resolve in microseconds
 
 ## Use Cases
 
@@ -412,7 +535,7 @@ from taxdumpy import (
 ```bash
 git clone https://github.com/omegahh/taxdumpy.git
 cd taxdumpy
-pip install -e .[dev]
+pip install -e '.[dev]'
 ```
 
 ### Testing
@@ -439,12 +562,21 @@ ruff check src/ tests/ --fix
 
 ### Release Process
 
+A pushed `v*` tag creates a GitHub Release. It does not build or publish a
+package to PyPI. Build and upload separately when publishing a release:
+
 ```bash
-# Automated release
-python scripts/release.py --version 1.2.3 --upload
+python -m build
+python -m twine upload dist/*
+```
+
+The release helper requires a clean worktree, runs tests and Ruff, updates and
+commits the version, pushes `origin main` and a version tag, removes `dist/`,
+and builds the package. `--upload` also publishes to PyPI. Run it only when
+those repository and remote changes are intended:
 
-# Or manual: create tag and push (GitHub Actions publishes to PyPI)
-git tag v1.2.3 && git push origin v1.2.3
+```bash
+python scripts/release.py --version 1.2.3 --upload
 ```
 
 ## License