annostat 1.0.0__tar.gz

annostat-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Annostat contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

annostat-1.0.0/PKG-INFO

@@ -0,0 +1,539 @@
+Metadata-Version: 2.4
+Name: annostat
+Version: 1.0.0
+Summary: Analyze bacterial genome annotations from GFF3 and FASTA files
+License: MIT License
+        
+        Copyright (c) 2026 Annostat contributors
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+        
+Project-URL: Repository, https://github.com/Karudhoru/annostat
+Project-URL: Issues, https://github.com/Karudhoru/annostat/issues
+Keywords: bioinformatics,genome-annotation,gff3,fasta,quality-control
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Science/Research
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Scientific/Engineering :: Bio-Informatics
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Dynamic: license-file
+
+# Annostat
+
+**Annostat** analyzes bacterial genome annotations from matching GFF3 and FASTA
+files. Local analysis uses only the Python standard library and supports multi-record FASTA files.
+
+## Features
+
+- Counts CDS, RNA types, all feature types, hypothetical proteins, gene annotations,
+  and CDS with COG categories
+- Exports a compact feature overview as CSV or TSV
+- Extracts every CDS on the correct strand as nucleotide and amino-acid multi-FASTA
+- Calculates codon usage percentages, start-codon counts, and individual COG category counts
+- Reports GC content, coding density, CDS overlaps and containment, pseudogenes,
+  partial features, structural-RNA completeness, frame issues, and ambiguous bases
+- Supports optional CDS exports filtered by length, COG annotation, and
+  hypothetical-protein status without changing the complete analysis
+- Builds a self-contained offline HTML report with provenance and performance timings
+- Saves three focused, dependency-free SVG charts for COG categories, CDS lengths,
+  and table-aware start codons; the COG chart is omitted when the source has no
+  COG data
+- Streams CDS processing and FASTA generation to reduce runtime and peak memory
+- Compares two or more local or NCBI-hosted annotations using normalized genome,
+  annotation-completeness, COG, start-codon, and QC profiles
+- Reports pairwise Jensen-Shannon profile distances and exact gene-symbol/COG-ID
+  overlap without presenting annotation labels as inferred orthologs
+- Downloads versioned RefSeq or GenBank assemblies through the official NCBI
+  Datasets command-line tool when network connectivity is requested
+- Separates deterministic FASTA/GFF3 validation from biological inspection, with
+  stable rule IDs, severity levels, schema/ruleset versions, and source citations
+- Records SHA-256 hashes for reproducibility and writes canonical JSON plus TSV
+  validation results suitable for CI pipelines
+- Correctly joins repeated-ID multipart CDS features in biological order on both
+  strands and validates phase continuity before translation
+- Supports NCBI prokaryotic translation tables 4, 11, and 25, automatically
+  selecting a consistent CDS or `region` `transl_table` declaration when present
+- Applies position-specific NCBI `transl_except` annotations for selenocysteine,
+  pyrrolysine, explicitly annotated termination codons, and `OTHER` residues
+  represented conservatively as `X`
+- Aggregates completed inspections into a script-free cohort HTML report,
+  normalized TSV matrix, and machine-readable JSON
+
+## Required and optional components
+
+| Component | Status | Purpose |
+|---|---|---|
+| Python 3.10 or newer | Required | Runs Annostat and its test suite |
+| Genome FASTA (`.fna`, `.fa`, or `.fasta`) | Required | Supplies the genome sequences |
+| Matching GFF3 annotation (`.gff` or `.gff3`) | Required | Supplies feature coordinates and annotations |
+| Matching FASTA/GFF3 sequence identifiers | Required | Connects every annotated feature to its sequence |
+| NCBI Datasets CLI | Optional | Downloads GCF/GCA assemblies for `fetch` or `compare --reference` |
+| Internet connection | Optional | Needed only when downloading NCBI data |
+
+No third-party Python package is required for local analysis, report generation,
+tables, FASTA exports, or SVG plots.
+
+## Core functionality
+
+Annostat provides the following core analysis outputs:
+
+- Counts CDS, RNA features by type, hypothetical CDS, CDS with gene names, CDS
+  with COG categories, and other feature types.
+- Writes a CSV or TSV feature overview containing ID, type, start, stop, gene,
+  product, and strand, with missing values left empty.
+- Exports all CDS as nucleotide and translated amino-acid multi-FASTA files.
+- Calculates pooled codon usage percentages, start-codon counts, and detailed
+  COG-category statistics, including multi-category assignments.
+- Generates at least two meaningful scientific plots; a normal run produces CDS
+  length, start-codon, and—when available—COG-category SVG figures.
+- Provides an `argparse` command-line interface and modular Python structure with
+  documented functions, classes, and methods.
+
+Filtering, quality control, HTML reports, performance profiling, comparative
+analysis, cohort summaries, and NCBI connectivity extend the core workflow.
+
+## Input assumptions and scope
+
+- GFF3 records must contain nine tab-separated fields with 1-based inclusive
+  coordinates. The FASTA identifier is the first whitespace-delimited header token.
+- CDS strand and phase are honored. Circular-origin CDS coordinates extending past
+  the sequence length are supported when a `region` feature declares
+  `Is_circular=true`.
+- Repeated CDS IDs are interpreted as one multipart biological feature. Segments
+  must use the same sequence ID, strand, and Parent; phase continuity is checked,
+  and segments are joined in biological order before translation.
+- COG coverage describes categories actually present in the annotation source. An
+  annotation with no COG category fields is reported as unavailable, not as 0%.
+- Annostat uses the consistent CDS or `region` `transl_table` declared in GFF3.
+  When the annotation declares no table, it falls back to bacterial table 11. `--genetic-code` is an
+  explicit override for annotations without that metadata; an override that
+  conflicts with a declared table is rejected instead of silently mistranslating
+  CDS features. Annostat never guesses a genetic code from sequence composition.
+
+## Installation
+
+Annostat requires Python 3.10 or newer and has no third-party runtime
+dependencies. Clone the repository and create an isolated environment.
+
+Linux or WSL:
+
+```bash
+git clone https://github.com/Karudhoru/annostat.git
+cd annostat
+python3 -m venv .venv
+source .venv/bin/activate
+```
+
+Windows PowerShell:
+
+```powershell
+git clone https://github.com/Karudhoru/annostat.git
+Set-Location annostat
+py -3 -m venv .venv
+.\.venv\Scripts\Activate.ps1
+```
+
+Install the current release from the checkout and verify the executable:
+
+```bash
+python -m pip install .
+annostat --version
+annostat --help
+```
+
+Expected version output:
+
+```text
+Annostat 1.0.0
+```
+
+Contributors should use `python -m pip install -e .` so source changes take
+effect without reinstalling. If the executable is unavailable, the equivalent
+module form is `python -m annostat`.
+
+## Quick-start guide
+
+### 1. Check the inputs
+
+Confirm that you have one genome FASTA and its matching GFF3 annotation. Sequence
+IDs must agree—for example, a FASTA record named `chromosome` must be referenced
+as `chromosome` in the first GFF3 column.
+
+### 2. Validate the paired files
+
+Run deterministic integrity checks before biological interpretation:
+
+```bash
+annostat validate \
+  -f data/GCF_000007145.1.fna \
+  -g data/GCF_000007145.1.gff3 \
+  -o validation
+```
+
+Example output for the included dataset:
+
+```text
+Annostat 1.0.0 | annotation validation PASS
+  Errors                   0
+  Warnings                 0
+  Findings                 0
+  Files written            2
+```
+
+The command writes `validation.json` and `validation.tsv`. Exit status `0` means
+no structural errors were found; status `1` means the configured threshold was
+reached. Use `--fail-on warning` for strict CI, or `--fail-on never` when the
+report should never stop a pipeline. A validation pass means the paired files are
+internally interpretable; it does not prove that every biological annotation is
+correct.
+
+### 3. Inspect one annotation
+
+```bash
+annostat inspect \
+  -f data/GCF_000007145.1.fna \
+  -g data/GCF_000007145.1.gff3 \
+  -o results \
+  --table-format tsv
+```
+
+Here `-f` selects the required FASTA, `-g` selects the required GFF3, and `-o`
+chooses the output directory. Add `--table-format tsv` when a TSV overview is
+preferred over CSV. The original command without the `inspect` word remains a
+backward-compatible alias. Translation-table selection is automatic when the
+GFF3 CDS or `region` rows contain `transl_table`; otherwise table 11 is used.
+The included example produces a 5,076,188 bp analysis with 4,295 CDS. For a local
+annotation without translation-table metadata, select a supported table explicitly
+with `--genetic-code 4`, `--genetic-code 11`, or `--genetic-code 25`.
+
+### 4. Review the result package
+
+Open `results/report.html` in a browser first. Then use:
+
+- `summary.json` for the complete machine-readable summary;
+- `tables/` for feature, codon, COG, start-codon, and QC data;
+- `sequences/` for nucleotide and protein CDS FASTA files; and
+- `plots/` for editable SVG figures.
+
+### 5. Summarize a cohort
+
+After inspecting several genomes into separate result directories, list those
+directories explicitly or pass their common parent directory:
+
+```bash
+annostat summarize results/sample-a results/sample-b -o cohort
+```
+
+The output contains `cohort.html`, `cohort.tsv`, and `cohort.json`. Samples are
+ordered deterministically. Missing source-dependent values such as COG coverage
+remain `NA`/`null`, never biological zero. Multiple source Annostat versions are
+retained and flagged in the report.
+
+## Common workflows
+
+The following sections show filtered exports, comparative analysis, NCBI
+references, performance profiling, and complete output descriptions.
+
+Without installing, module execution remains available from the repository root:
+`python -m annostat --help`.
+
+Use `--table-format tsv` for a tab-separated feature overview. Add `--profile`
+to measure peak Python memory and print per-stage timings.
+
+### Filtered CDS exports
+
+Filters create an additional `filtered/` directory while retaining every normal
+analysis file. Criteria are combined, so a CDS must satisfy all enabled options:
+
+```bash
+annostat \
+  -f genome.fna \
+  -g annotations.gff3 \
+  -o results \
+  --min-cds-length 300 \
+  --require-cog \
+  --exclude-hypothetical
+```
+
+Available criteria are `--min-cds-length`, `--max-cds-length`, `--require-cog`,
+and `--exclude-hypothetical`.
+
+### Comparative analysis
+
+Compare two or more labelled local genomes with repeatable `--genome` options:
+
+```bash
+annostat compare \
+  --genome genome-a data/GCF_000007145.1.fna data/GCF_000007145.1.gff3 \
+  --genome genome-b data/GCF_001050915.2.fna data/GCF_001050915.2.gff3 \
+  --output comparison
+```
+
+Each genome is structurally validated and independently uses its declared
+`transl_table` or the table-11 fallback. For an annotation without
+`transl_table`, a per-genome override uses
+the same label supplied to `--genome` (or the reference accession):
+
+```bash
+annostat compare \
+  --genome sr1 sr1.fna sr1.gff3 \
+  --genome ecoli ecoli.fna ecoli.gff3 \
+  --genetic-code sr1 25 \
+  --output comparison
+```
+
+The comparison package is intentionally compact:
+
+```text
+comparison/
+|-- comparison.html
+|-- comparison.json
+|-- tables/
+|   |-- dataset_metrics.tsv
+|   `-- pairwise_comparisons.tsv
+`-- plots/
+    |-- dataset_overview.svg
+    `-- cog_comparison.svg (only when every input supplies COG categories)
+```
+
+`dataset_metrics.tsv` provides raw genome/feature counts together with GC,
+coding density, CDS per Mb, annotation coverage, start-codon usage, and QC
+warnings. `pairwise_comparisons.tsv` contains metric deltas, percentage deltas,
+Jensen-Shannon distances for COG and start-codon profiles, and Jaccard overlap
+for exact gene symbols and explicit COG identifiers.
+
+For two datasets, the COG figure shows percentage-point differences. For three
+or more datasets, it becomes a dataset-by-category heatmap. Exact gene-symbol
+overlap describes annotation labels only; it is not orthology, ANI, phylogeny,
+or a core/accessory pangenome analysis.
+
+### Compare against external NCBI data
+
+NCBI access is optional and uses NCBI's maintained `datasets` executable. Follow
+the official [installation instructions](https://www.ncbi.nlm.nih.gov/datasets/docs/v2/command-line-tools/download-and-install/),
+then add an external reference directly to a local comparison:
+
+```bash
+annostat compare \
+  --genome local-sample sample.fna sample.gff3 \
+  --reference GCF_000007145.1 \
+  --output comparison
+```
+
+`--reference` is repeatable, so a comparison may contain only external
+assemblies or any mixture of local and NCBI inputs. Annostat downloads genome
+FASTA, GFF3, and assembly metadata under `comparison/external_inputs/`, then
+runs every input through the same comparison engine:
+
+```bash
+annostat compare \
+  --reference GCF_000007145.1 \
+  --reference GCF_001050915.2 \
+  --output comparison
+```
+
+The report identifies organisms and whether each pair belongs to the same or
+different species when the input metadata makes that determination possible.
+It never assumes that labelled datasets are strains. NCBI assembly names,
+taxonomy identifiers, annotation providers, and release metadata are retained
+in `comparison.json`. Versioned GCF/GCA accessions in the HTML provenance table
+link to their NCBI Datasets assembly pages.
+
+COG results are source-dependent. Bakta annotations commonly include COG
+categories, while current NCBI RefSeq GFF3 packages may not. When a source does
+not supply COG fields, Annostat reports COG coverage and pairwise COG statistics
+as `Not available` and omits the COG comparison plot; missing data is never
+presented as biological zero coverage.
+
+For users who only need the source files, the standalone downloader remains
+available:
+
+```bash
+annostat fetch GCF_000007145.1 GCF_001050915.2 --output ncbi_data
+```
+
+The `NCBI_API_KEY` environment variable is honored by the official client. Local
+analysis never requires a network connection, and Annostat does not transmit
+local FASTA or GFF3 files.
+
+The output directory contains:
+
+```text
+results/
+├── report.html
+├── summary.json
+├── tables/
+│   ├── features.csv (or features.tsv)
+│   ├── codon_usage.csv
+│   ├── start_codons.csv
+│   ├── cog_categories.csv
+│   └── annotation_issues.csv
+├── validation/
+│   ├── validation.json
+│   └── validation.tsv
+├── sequences/
+│   ├── cds_nucleotide.fasta
+│   └── cds_protein.fasta
+└── plots/
+    ├── cog_categories.svg (when COG categories are available)
+    ├── cds_lengths.svg
+    └── start_codons.svg
+```
+
+When filters are active, `filtered/` additionally contains the selected feature
+table and paired nucleotide/protein FASTA files.
+
+`report.html` works without a server or internet connection and embeds the SVG
+figures directly. The separate SVG files remain available for editing and use in
+documents. Codon usage remains available as the required percentage table; a
+dense 64-codon heatmap is intentionally not generated by default.
+
+`annotation_issues.csv` is a review aid rather than an automatic pass/fail
+verdict. Short CDS overlaps are common in compact bacterial genomes, so ordinary
+overlaps and adjacent duplicate labels are informational. Containment, incomplete
+frames, missing structural RNA classes, internal stops, and ambiguous bases are
+reported as warnings that merit inspection in their biological context.
+
+## Validation versus inspection
+
+`annostat validate` answers whether the files can be interpreted consistently:
+GFF3 syntax, FASTA symbols, matching sequence IDs, coordinate bounds, Parent/ID
+relationships, multipart-feature consistency, CDS phase continuity, and valid,
+consistent CDS translation-table declarations. These checks are deterministic
+and may safely control a CI exit status.
+
+`annostat inspect` first requires structural validation to pass, then reports
+biological review candidates such as internal stops, missing terminal codons,
+ambiguous CDS bases, overlaps, partial features, pseudogenes, duplicated adjacent
+labels, and structural-RNA coverage. These findings are context-dependent and do
+not automatically make an annotation invalid. Boundary-specific partial markers
+suppress only the affected start- or stop-codon assumption. Explicit `pseudo`,
+`transl_except`, and `exception` annotations suppress checks whose assumptions do
+not apply.
+
+Every validation result embeds its rule definitions, rationale, source URL,
+ruleset version, schema version, Annostat version, and input SHA-256 hashes. JSON
+and TSV findings use canonical ordering and contain no timestamp, so repeated runs
+on identical paths and bytes produce byte-identical validation artifacts. A
+`scientific_fingerprint` in both validation and inspection JSON excludes file
+paths and performance timings, allowing scientific results to be compared across
+machines and output directories.
+
+## Testing and verification
+
+Run compilation and all automated tests with the interpreter used to install
+Annostat:
+
+```bash
+python -m compileall -q annostat tests
+python -m unittest discover -v
+```
+
+For a final end-to-end check, analyze one supplied genome and confirm that
+`results/report.html`, `results/summary.json`, the tables, FASTA exports, and SVG
+plots are created:
+
+```bash
+annostat inspect \
+  -f data/GCF_000007145.1.fna \
+  -g data/GCF_000007145.1.gff3 \
+  -o results \
+  --table-format tsv
+```
+
+## Example run
+
+```bash
+annostat inspect \
+  -f data/GCF_000007145.1.fna \
+  -g data/GCF_000007145.1.gff3 \
+  -o results \
+  --table-format tsv
+```
+
+Example output:
+
+```text
+Annostat 1.0.0 | bacterial genome annotation analysis
+FASTA: /path/to/annostat/data/GCF_000007145.1.fna
+GFF3:  /path/to/annostat/data/GCF_000007145.1.gff3
+
+[1/5] Reading GFF3 annotations and FASTA sequences
+[2/5] Streaming, translating, and writing CDS sequences
+[3/5] Calculating statistics and writing analysis tables
+[4/5] Rendering scientific visualizations
+[5/5] Building the offline HTML report
+
+Analysis summary
+--------------------------------------------------------
+  Genome size                              5,076,188 bp
+  Sequences                              1 (1 circular)
+  Features                                        4,498
+  CDS                                             4,295
+  RNA features                                      181
+  Genome GC                                      65.07%
+  Coding density                                 85.33%
+  Hypothetical CDS                         619 (14.41%)
+  COG-annotated CDS                      2,881 (67.08%)
+  Recognized starts                      4,294 (99.98%)
+  QC review                1 warning, 872 informational findings
+--------------------------------------------------------
+  Output                   /path/to/annostat/results
+  Files written            14
+  Completed in             0.56 seconds
+```
+
+Runtime depends on the input size and system. The supplied bacterial genome
+typically completes in under a second on a modern desktop. Use `--profile` when
+you need measured stage timings and peak-memory information; profiling itself
+adds overhead and should not be used for speed benchmarks.
+
+## Scientific basis
+
+The QC checks follow the internal-consistency emphasis of the
+[NCBI Prokaryotic Genome Annotation Standards](https://www.ncbi.nlm.nih.gov/refseq/annotation_prok/standards/)
+and the attribute conventions in the
+[NCBI GFF3 documentation](https://www.ncbi.nlm.nih.gov/datasets/docs/v2/reference-docs/file-formats/annotation-files/about-ncbi-gff3/).
+FASTA validation accepts the NCBI/IUPAC nucleotide alphabet, and translation
+tables and initiator codons follow the
+[NCBI Genetic Codes](https://www.ncbi.nlm.nih.gov/Taxonomy/Utils/wprintgc.cgi).
+The report treats overlaps as review candidates because overlapping genes are
+common in compact bacterial genomes. Translation uses the bacterial code and
+alternative-start behavior described by
+[Prodigal](https://pmc.ncbi.nlm.nih.gov/articles/PMC2848648/). The project remains
+an annotation-analysis tool rather than a replacement for comprehensive
+annotation pipelines such as
+[Bakta](https://pmc.ncbi.nlm.nih.gov/articles/PMC8743544/) or
+[Prokka](https://pubmed.ncbi.nlm.nih.gov/24642063/).
+
+Comparative profiles are normalized to avoid treating larger genomes as
+automatically enriched. Full ortholog and pangenome inference remains the domain
+of tools such as [Panaroo](https://pmc.ncbi.nlm.nih.gov/articles/PMC7376924/),
+while sequence-level average nucleotide identity should be calculated by a
+specialized implementation such as
+[FastANI](https://pmc.ncbi.nlm.nih.gov/articles/PMC6269478/).