catomatic 0.2.0__tar.gz → 0.2.3__tar.gz

{catomatic-0.2.0/src/catomatic.egg-info → catomatic-0.2.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: catomatic
-Version: 0.2.0
+Version: 0.2.3
 Summary: A tool for automatically building catalogues of antibiotic resistance-associated variants
 Author: Dylan Adlard
 Author-email: Philip W Fowler <philip.fowler@ndm.ox.ac.uk>
@@ -48,15 +48,15 @@ The first is used in [https://doi.org/10.1101/2025.01.30.635633](https://doi.org
 
 This method relies on the logic that mutations that do not cause resistance can co-occur with those that do. If a mutation in isolation (solo) does not cause resistance, then it is not contributing to the phenotype when present in a mixture either.
 
-Mutations that occur in isolation across specified genes are traversed in sequence, and if their proportion of drug-susceptibility (vs. resistance) passes the specified statistical test, they are characterized as benign (susceptible) and removed from the dataset. This step repeats iteratively until no more solo benign mutations are found.
+Mutations that occur in isolation are traversed in sequence. When they meet the susceptible criteria for the selected test, they are characterized as benign (`S`) and removed from the dataset. This step repeats iteratively until no more solo benign mutations are found.
 
 Remaining mutations are classified based on their resistance rates and statistical test results. Those that don’t meet thresholds are labeled as `U`.
 
 The classification approach supports:
 
-- **No test**: assumes homogeneous susceptibility is sufficient for S
-- **Binomial test**: against a specified background resistance rate
-- **Fisher's test**: using a contingency background
+- **No test**: calls a mutation `S` during iterative unlocking when all its solo observations are susceptible; remaining solo mutations are `U`
+- **Binomial test**: tests the solo resistance rate against a specified background resistance rate
+- **Fisher's test**: compares the mutation solo contingency table with the no-mutation solo background
 
 ### Optional Interventions
 
@@ -68,9 +68,9 @@ Because the method uses GARC1 grammar, rules like `{rpoB@*_fs:R}` can be supplie
 - Be additive (lower priority than specific entries)
 - Replace matching mutations (requires `replace=True` and `wildcards` supplied)
 
-The catalogue can be returned as a dictionary or a Piezo-compatible `pandas.DataFrame`.
+The built catalogue is available as a dictionary through the `catalogue.catalogue` attribute. It can also be converted to a Piezo-compatible `pandas.DataFrame` with `build_piezo()`.
 
-Contingency tables, proportions, p-values, and Wilson confidence intervals are stored in the `EVIDENCE` field.
+Evidence is stored with each catalogue entry and exported in the `EVIDENCE` field. It includes proportions, Wilson confidence intervals, solo contingency tables used for classification, full sample-level contingency tables for description, and p-values when a statistical test is used. Sample IDs can be included with `record_ids=True`.
 
 ### Example Workflow using Fisher's:
 
@@ -125,7 +125,7 @@ from catomatic.BinaryCatalogue import BinaryBuilder
 catalogue = BinaryBuilder(samples=samples_df, mutations=mutations_df).build()
 
 # View dictionary version
-cat_dict = catalogue.return_catalogue()
+cat_dict = catalogue.catalogue
 
 # Convert to Piezo-compatible format
 catalogue_df = catalogue.build_piezo(
@@ -149,7 +149,7 @@ catalogue.to_piezo(
 
 ### CLI
 
-After installation, the simplest way to run the catomatic catalogue builder is via the command line interface using the `binary` subcommand. You must use either `--to_piezo` or `--to_json` to specify the output format. Additional metadata is required when using `--to_piezo`.
+After installation, the simplest way to run the catomatic catalogue builder is via the command line interface using the `binary` subcommand. Use `--to_piezo` or `--to_json` to export the catalogue. If neither is supplied, no file is exported. Additional metadata is required when using `--to_piezo`.
 
 #### Export to JSON
 
@@ -178,6 +178,8 @@ catomatic binary \
 
 ### CLI Parameters
 
+The Binary CLI does not expose the Python `build()` options `tails` or `min_count`. The output flags are not mutually exclusive; if both are supplied, both exporters write to the same `--outfile`, so use only one.
+
 | Parameter          | Type    | Description                                                                                    |
 | ------------------ | ------- | ---------------------------------------------------------------------------------------------- |
 | `--samples`        | `str`   | Path to the samples (phenotypes) file. Required.                                               |
@@ -190,11 +192,16 @@ catomatic binary \
 | `--version`        | `str`   | Catalogue version. Required with `--to_piezo`.                                                 |
 | `--drug`           | `str`   | Name of the drug. Required with `--to_piezo`.                                                  |
 | `--wildcards`      | `str`   | Path to JSON file containing wildcard mutation definitions. Required with `--to_piezo`.        |
-| `--test`           | `str`   | Type of statistical test to apply. One of: `None`, `Binomial`, or `Fisher`. Optional.          |
-| `--background`     | `float` | Background mutation rate (0–1). Required if `--test Binomial` is used.                         |
-| `--p`              | `float` | P-value threshold for statistical significance. Optional. Defaults to `0.95`.                  |
-| `--tails`          | `str`   | Tail type for statistical test. One of: `one`, `two`. Optional. Defaults to `two`.             |
-| `--strict_unlock`  | `flag`  | If set, disables classification of susceptible (`S`) mutations unless statistically confident. |
+| `--test`           | `str`   | Type of statistical test to apply: `Binomial` or `Fisher`. Optional; if omitted, no statistical test is used. |
+| `--background`     | `float` | Background resistance rate (0–1). Required if `--test Binomial` is used.                       |
+| `--p`              | `float` | Confidence level used for statistical testing. Optional. Defaults to `0.95` (a 0.05 significance threshold). |
+| `--strict_unlock`  | `flag`  | For two-tailed tests, require statistical significance to classify homogeneous susceptible mutations as `S` during iterative unlocking. |
+| `--frs`            | `float` | Optional fraction-read-support threshold. Requires an `FRS` column in the mutations file.       |
+| `--seed`           | `str[]`  | Optional list of mutations to seed as susceptible before building.                              |
+| `--record_ids`     | `flag`  | Include sample IDs in evidence records.                                                        |
+| `--grammar`        | `str`   | Grammar used for Piezo export. Default: `GARC1`.                                                |
+| `--values`         | `str`   | Prediction values for Piezo export. Default: `RUS`.                                             |
+| `--for_piezo`      | `flag`  | Add placeholder rows for Piezo export. Default: `False` in the CLI.                            |
 
 ## Running catomatic's Regression Builder
 
@@ -310,8 +317,8 @@ catomatic regression \
 
 ### Notes
 
-- When using post-hoc rule updates via .update(), you must provide wildcards and set replace=True if you intend to override existing entries.
-- For Piezo export, placeholder entries are inserted automatically if needed to satisfy parser requirements (R, S, and U must be represented).
+- For `BinaryBuilder`, use `update_catalogue()` to apply post-hoc rules. Supply `wildcards` and set `replace=True` to replace entries matched by an aggregate rule. Specific entries can be added or replaced without `replace=True`.
+- For Piezo export, placeholder entries are inserted when `for_piezo=True` and needed to satisfy parser requirements (R, S, and U must be represented). The Python export methods default to `for_piezo=True`; the CLI flag is off by default.
 - The EVIDENCE column includes contingency tables, proportions, confidence intervals, and p-values, and may optionally include sample IDs if `record_ids=True`.
 - To build a catalogue with the regression builder, as currently implemented, requires an ecoff as it will compare the predited effected against the background to supply an R/S/U label
   - To only calculate predicted effects, this can be done in Python by calling RegressionBuilder.predict_effects()

{catomatic-0.2.0 → catomatic-0.2.3}/README.md

@@ -23,15 +23,15 @@ The first is used in [https://doi.org/10.1101/2025.01.30.635633](https://doi.org
 
 This method relies on the logic that mutations that do not cause resistance can co-occur with those that do. If a mutation in isolation (solo) does not cause resistance, then it is not contributing to the phenotype when present in a mixture either.
 
-Mutations that occur in isolation across specified genes are traversed in sequence, and if their proportion of drug-susceptibility (vs. resistance) passes the specified statistical test, they are characterized as benign (susceptible) and removed from the dataset. This step repeats iteratively until no more solo benign mutations are found.
+Mutations that occur in isolation are traversed in sequence. When they meet the susceptible criteria for the selected test, they are characterized as benign (`S`) and removed from the dataset. This step repeats iteratively until no more solo benign mutations are found.
 
 Remaining mutations are classified based on their resistance rates and statistical test results. Those that don’t meet thresholds are labeled as `U`.
 
 The classification approach supports:
 
-- **No test**: assumes homogeneous susceptibility is sufficient for S
-- **Binomial test**: against a specified background resistance rate
-- **Fisher's test**: using a contingency background
+- **No test**: calls a mutation `S` during iterative unlocking when all its solo observations are susceptible; remaining solo mutations are `U`
+- **Binomial test**: tests the solo resistance rate against a specified background resistance rate
+- **Fisher's test**: compares the mutation solo contingency table with the no-mutation solo background
 
 ### Optional Interventions
 
@@ -43,9 +43,9 @@ Because the method uses GARC1 grammar, rules like `{rpoB@*_fs:R}` can be supplie
 - Be additive (lower priority than specific entries)
 - Replace matching mutations (requires `replace=True` and `wildcards` supplied)
 
-The catalogue can be returned as a dictionary or a Piezo-compatible `pandas.DataFrame`.
+The built catalogue is available as a dictionary through the `catalogue.catalogue` attribute. It can also be converted to a Piezo-compatible `pandas.DataFrame` with `build_piezo()`.
 
-Contingency tables, proportions, p-values, and Wilson confidence intervals are stored in the `EVIDENCE` field.
+Evidence is stored with each catalogue entry and exported in the `EVIDENCE` field. It includes proportions, Wilson confidence intervals, solo contingency tables used for classification, full sample-level contingency tables for description, and p-values when a statistical test is used. Sample IDs can be included with `record_ids=True`.
 
 ### Example Workflow using Fisher's:
 
@@ -100,7 +100,7 @@ from catomatic.BinaryCatalogue import BinaryBuilder
 catalogue = BinaryBuilder(samples=samples_df, mutations=mutations_df).build()
 
 # View dictionary version
-cat_dict = catalogue.return_catalogue()
+cat_dict = catalogue.catalogue
 
 # Convert to Piezo-compatible format
 catalogue_df = catalogue.build_piezo(
@@ -124,7 +124,7 @@ catalogue.to_piezo(
 
 ### CLI
 
-After installation, the simplest way to run the catomatic catalogue builder is via the command line interface using the `binary` subcommand. You must use either `--to_piezo` or `--to_json` to specify the output format. Additional metadata is required when using `--to_piezo`.
+After installation, the simplest way to run the catomatic catalogue builder is via the command line interface using the `binary` subcommand. Use `--to_piezo` or `--to_json` to export the catalogue. If neither is supplied, no file is exported. Additional metadata is required when using `--to_piezo`.
 
 #### Export to JSON
 
@@ -153,6 +153,8 @@ catomatic binary \
 
 ### CLI Parameters
 
+The Binary CLI does not expose the Python `build()` options `tails` or `min_count`. The output flags are not mutually exclusive; if both are supplied, both exporters write to the same `--outfile`, so use only one.
+
 | Parameter          | Type    | Description                                                                                    |
 | ------------------ | ------- | ---------------------------------------------------------------------------------------------- |
 | `--samples`        | `str`   | Path to the samples (phenotypes) file. Required.                                               |
@@ -165,11 +167,16 @@ catomatic binary \
 | `--version`        | `str`   | Catalogue version. Required with `--to_piezo`.                                                 |
 | `--drug`           | `str`   | Name of the drug. Required with `--to_piezo`.                                                  |
 | `--wildcards`      | `str`   | Path to JSON file containing wildcard mutation definitions. Required with `--to_piezo`.        |
-| `--test`           | `str`   | Type of statistical test to apply. One of: `None`, `Binomial`, or `Fisher`. Optional.          |
-| `--background`     | `float` | Background mutation rate (0–1). Required if `--test Binomial` is used.                         |
-| `--p`              | `float` | P-value threshold for statistical significance. Optional. Defaults to `0.95`.                  |
-| `--tails`          | `str`   | Tail type for statistical test. One of: `one`, `two`. Optional. Defaults to `two`.             |
-| `--strict_unlock`  | `flag`  | If set, disables classification of susceptible (`S`) mutations unless statistically confident. |
+| `--test`           | `str`   | Type of statistical test to apply: `Binomial` or `Fisher`. Optional; if omitted, no statistical test is used. |
+| `--background`     | `float` | Background resistance rate (0–1). Required if `--test Binomial` is used.                       |
+| `--p`              | `float` | Confidence level used for statistical testing. Optional. Defaults to `0.95` (a 0.05 significance threshold). |
+| `--strict_unlock`  | `flag`  | For two-tailed tests, require statistical significance to classify homogeneous susceptible mutations as `S` during iterative unlocking. |
+| `--frs`            | `float` | Optional fraction-read-support threshold. Requires an `FRS` column in the mutations file.       |
+| `--seed`           | `str[]`  | Optional list of mutations to seed as susceptible before building.                              |
+| `--record_ids`     | `flag`  | Include sample IDs in evidence records.                                                        |
+| `--grammar`        | `str`   | Grammar used for Piezo export. Default: `GARC1`.                                                |
+| `--values`         | `str`   | Prediction values for Piezo export. Default: `RUS`.                                             |
+| `--for_piezo`      | `flag`  | Add placeholder rows for Piezo export. Default: `False` in the CLI.                            |
 
 ## Running catomatic's Regression Builder
 
@@ -285,8 +292,8 @@ catomatic regression \
 
 ### Notes
 
-- When using post-hoc rule updates via .update(), you must provide wildcards and set replace=True if you intend to override existing entries.
-- For Piezo export, placeholder entries are inserted automatically if needed to satisfy parser requirements (R, S, and U must be represented).
+- For `BinaryBuilder`, use `update_catalogue()` to apply post-hoc rules. Supply `wildcards` and set `replace=True` to replace entries matched by an aggregate rule. Specific entries can be added or replaced without `replace=True`.
+- For Piezo export, placeholder entries are inserted when `for_piezo=True` and needed to satisfy parser requirements (R, S, and U must be represented). The Python export methods default to `for_piezo=True`; the CLI flag is off by default.
 - The EVIDENCE column includes contingency tables, proportions, confidence intervals, and p-values, and may optionally include sample IDs if `record_ids=True`.
 - To build a catalogue with the regression builder, as currently implemented, requires an ecoff as it will compare the predited effected against the background to supply an R/S/U label
   - To only calculate predicted effects, this can be done in Python by calling RegressionBuilder.predict_effects()

{catomatic-0.2.0 → catomatic-0.2.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "catomatic"
-version = "0.2.0"
+version = "0.2.3"
 description = "A tool for automatically building catalogues of antibiotic resistance-associated variants"
 readme = "README.md"
 # If README.md is markdown, setuptools will pick it up; optionally:

{catomatic-0.2.0 → catomatic-0.2.3}/src/catomatic/BinaryCatalogue.py

@@ -27,7 +27,7 @@ class BinaryBuilder(PiezoExporter):
                                   Required columns: ['UNIQUEID', 'MUTATION']
                                   Optional columns: ['FRS']
 
-        FRS (float, optional): The Fraction Read Support threshold used to construct the catalogues.
+        frs (float, optional): The Fraction Read Support threshold used to construct the catalogues.
                                Lower FRS values allow for greater genotype heterogeneity.
 
         seed (list) optional): A list of predefined GARC neutral mutations with associated phenotypes
@@ -140,10 +140,36 @@ class BinaryBuilder(PiezoExporter):
         # If no more susceptible solos, classify all R and U solos in one, final sweep
         self.classify(self.samples, self.mutations)
 
+        # add in rows for mutations without solos
+        # self.add_missing()
+
         self.order_catalogue()
 
         return self
 
+    def add_missing(self):
+
+        # add in remaining mutations
+        all_muts = set(self.mutations.MUTATION.dropna().unique())
+        classified_muts = set(self.catalogue.keys())
+        missing = all_muts - classified_muts
+
+        for mut in missing:
+            full_x = self.build_full_contingency(self.samples, self.mutations, mut)
+
+            proportion = self.calc_proportion(full_x)
+            ci = self.calc_confidence_interval(full_x)
+
+            data = {
+                "proportion": proportion,
+                "confidence": ci,
+                "full_contingency": full_x,
+                "solo_contingency": None,  # explicitly absent
+                "note": "No solo evidence; added post hoc",
+            }
+
+            self.add_mutation(mut, "U", data)
+
     def classify(self, samples: pd.DataFrame, mutations: pd.DataFrame) -> None:
         """
         Orchestrate one classification iteration over exposed 'solo' mutations.
@@ -196,22 +222,26 @@ class BinaryBuilder(PiezoExporter):
         # Skip mutations with fewer than min_count samples
         if solos[solos.MUTATION == mut].shape[0] < self.min_count:
             return
-        # build a contingency table
-        x, ids = self.build_contingency(solos, mut)
+        # build a contingency table for solo mutations
+        x, ids = self.build_solo_contingency(solos, mut)
+        # build a contingency table for all observations
+        full_x = self.build_full_contingency(self.samples, self.mutations, mut)
         # temporarily store mutation groups:
         self.temp_ids = ids
 
         # classify susceptible variants according to specified test mode
         if self.test is None:
-            self.skeleton_build(mut, x)
+            self.skeleton_build(mut, x, full_x)
         elif self.test == "Binomial":
-            self.binomial_build(mut, x)
+            self.binomial_build(mut, x, full_x)
         elif self.test == "Fisher":
-            self.fishers_build(mut, x)
+            self.fishers_build(mut, x, full_x)
         else:
             raise ValueError(f"Unknown test mode: {self.test}")
 
-    def skeleton_build(self, mutation: str, x: Contingency) -> None:
+    def skeleton_build(
+        self, mutation: str, x: Contingency, full_x: Contingency
+    ) -> None:
         """
         Record descriptive statistics and optionally mark susceptible solos.
         Calls homogenous susceptible S.
@@ -227,7 +257,12 @@ class BinaryBuilder(PiezoExporter):
         proportion = self.calc_proportion(x)
         ci = self.calc_confidence_interval(x)
 
-        data = {"proportion": proportion, "confidence": ci, "contingency": x}
+        data = {
+            "proportion": proportion,
+            "confidence": ci,
+            "solo_contingency": x,
+            "contingency": full_x,
+        }
 
         if self.run_iter:
             # if iteratively classifing S variants
@@ -238,8 +273,12 @@ class BinaryBuilder(PiezoExporter):
             # not phenotyping, just adding to catalogue
             self.add_mutation(mutation, "U", data)
 
-    def binomial_build(self, mutation: str, x: Contingency) -> None:
-        assert self.background is not None, "background must be provided for Binomial test"
+    def binomial_build(
+        self, mutation: str, x: Contingency, full_x: Contingency
+    ) -> None:
+        assert (
+            self.background is not None
+        ), "background must be provided for Binomial test"
         bg: float = float(self.background)
 
         # p-value function for binomial
@@ -258,10 +297,11 @@ class BinaryBuilder(PiezoExporter):
         def resistant_rule(proportion: float, p_calc: float, x) -> bool:
             return bool(proportion > bg)
 
-        self.hypothesis_test(mutation, x, pvalue_fn, susceptible_rule, resistant_rule)
-
+        self.hypothesis_test(
+            mutation, x, full_x, pvalue_fn, susceptible_rule, resistant_rule
+        )
 
-    def fishers_build(self, mutation: str, x: Contingency) -> None:
+    def fishers_build(self, mutation: str, x: Contingency, full_x: Contingency) -> None:
         """
         Classify mutation using Fisher's exact test and directional inference.
 
@@ -291,12 +331,15 @@ class BinaryBuilder(PiezoExporter):
             odds = self.calc_odds_ratio(x)
             return odds > 1
 
-        self.hypothesis_test(mutation, x, pvalue_fn, susceptible_rule, resistant_rule)
+        self.hypothesis_test(
+            mutation, x, full_x, pvalue_fn, susceptible_rule, resistant_rule
+        )
 
     def hypothesis_test(
         self,
         mutation: str,
         x: Contingency,
+        full_x: Contingency,
         pvalue_fn: Callable[[Contingency], float],
         susceptible_rule: Callable[[float, float, Contingency], bool],
         resistant_rule: Callable[[float, float, Contingency], bool],
@@ -324,7 +367,8 @@ class BinaryBuilder(PiezoExporter):
             "proportion": proportion,
             "confidence": ci,
             "p_value": p_calc,
-            "contingency": x,
+            "solo_contingency": x,  # solo-based (classification)
+            "contingency": full_x,  #         (descriptive only)
         }
 
         # ITERATIVE MODE (we actively try to find susceptibles)
@@ -414,7 +458,7 @@ class BinaryBuilder(PiezoExporter):
         return (lower, upper)
 
     @staticmethod
-    def build_contingency(
+    def build_solo_contingency(
         solos: pd.DataFrame, mutation: str
     ) -> Tuple[list[list[int]], list[str]]:
         """
@@ -438,6 +482,30 @@ class BinaryBuilder(PiezoExporter):
 
         return [[R_count, S_count], [R_count_no_mut, S_count_no_mut]], ids
 
+    @staticmethod
+    def build_full_contingency(
+        samples: pd.DataFrame, mutations: pd.DataFrame, mutation: str
+    ) -> list[list[int]]:
+        """
+        Full contingency using ALL samples (sample-level, not row-level).
+        """
+
+        # determine which samples have the mutation
+        mut_present = mutations[mutations.MUTATION == mutation][
+            "UNIQUEID"
+        ].drop_duplicates()
+
+        df = samples.copy()
+        df["has_mut"] = df["UNIQUEID"].isin(mut_present)
+
+        R_with = ((df.PHENOTYPE == "R") & (df.has_mut)).sum()
+        S_with = ((df.PHENOTYPE == "S") & (df.has_mut)).sum()
+
+        R_without = ((df.PHENOTYPE == "R") & (~df.has_mut)).sum()
+        S_without = ((df.PHENOTYPE == "S") & (~df.has_mut)).sum()
+
+        return [[int(R_with), int(S_with)], [int(R_without), int(S_without)]]
+
     @staticmethod
     def calc_odds_ratio(x: Contingency) -> float:
         """