indico_toolkit 6.1.0__tar.gz → 6.14.0__tar.gz

indico_toolkit-6.14.0/PKG-INFO

@@ -0,0 +1,144 @@
+Metadata-Version: 2.3
+Name: indico-toolkit
+Version: 6.14.0
+Summary: Classes, functions, and abstractions for Indico IPA
+License: MIT
+Author: Indico Solutions Engineering
+Author-email: solutionsengineering@indicodata.ai
+Requires-Python: >=3.10
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Provides-Extra: all
+Provides-Extra: downloads
+Provides-Extra: examples
+Provides-Extra: metrics
+Provides-Extra: predictions
+Provides-Extra: snapshots
+Requires-Dist: indico-client (>=6.14.0,<7.0.0)
+Requires-Dist: pandas (>=2.2.3,<3.0.0) ; extra == "all"
+Requires-Dist: pandas (>=2.2.3,<3.0.0) ; extra == "downloads"
+Requires-Dist: pandas (>=2.2.3,<3.0.0) ; extra == "examples"
+Requires-Dist: pandas (>=2.2.3,<3.0.0) ; extra == "metrics"
+Requires-Dist: pandas (>=2.2.3,<3.0.0) ; extra == "predictions"
+Requires-Dist: pandas (>=2.2.3,<3.0.0) ; extra == "snapshots"
+Requires-Dist: plotly (>=5.2.1,<6.0.0) ; extra == "all"
+Requires-Dist: plotly (>=5.2.1,<6.0.0) ; extra == "metrics"
+Requires-Dist: tqdm (>=4.50.0,<5.0.0) ; extra == "all"
+Requires-Dist: tqdm (>=4.50.0,<5.0.0) ; extra == "downloads"
+Project-URL: source, https://github.com/IndicoDataSolutions/Indico-Solutions-Toolkit
+Description-Content-Type: text/markdown
+
+# Indico Toolkit
+
+**This repository contains software that is not officially supported by Indico. It may
+  be outdated or contain bugs. The operations it performs are potentially destructive.
+  Use at your own risk.**
+
+Classes, functions, and abstractions for building workflows using the Indico IPA
+(Intelligent Process Automation) platform.
+
+- [Polling Classes](https://github.com/IndicoDataSolutions/indico-toolkit-python/tree/main/indico_toolkit/polling/__init__.py)
+  that implement best-practices polling behavior for Auto Review and Downstream
+  processes. Easily plug in business logic without the boilerplate.
+- [Result File](https://github.com/IndicoDataSolutions/indico-toolkit-python/blob/main/indico_toolkit/results/__init__.py)
+  and [Etl Output](https://github.com/IndicoDataSolutions/indico-toolkit-python/blob/main/indico_toolkit/etloutput/__init__.py)
+  Data Classes that parse standard IPA JSON output into idiomatic, type-safe Python dataclasses.
+- [Metrics Classes](https://github.com/IndicoDataSolutions/indico-toolkit-python/blob/main/indico_toolkit/metrics/__init__.py)
+  to compare model performance, evaluate ground truth, and plot statistics.
+- [Snapshot Classes](https://github.com/IndicoDataSolutions/indico-toolkit-python/blob/main/indico_toolkit/snapshots/snapshot.py)
+  to concatenate, merge, filter, and manipulate snapshot CSVs.
+
+...and more in the [Examples](https://github.com/IndicoDataSolutions/indico-toolkit-python/tree/main/examples) folder.
+
+
+## Installation
+
+**Indico Toolkit does not use semantic versioning.**
+
+Indico Toolkit versions match the minimum IPA version required to use its functionality.
+E.g. `indico-toolkit==6.14.0` makes use of functionality introduced in IPA 6.14, and
+some functionality requires IPA 6.14 or later to use.
+
+```bash
+pip install indico-toolkit
+```
+
+Some functionality requires optional dependencies that can be installed with extras.
+
+```bash
+pip install 'indico-toolkit[all]'
+pip install 'indico-toolkit[downloads]'
+pip install 'indico-toolkit[examples]'
+pip install 'indico-toolkit[metrics]'
+pip install 'indico-toolkit[predictions]'
+pip install 'indico-toolkit[snapshots]'
+```
+
+
+## Contributing
+
+Indico Toolkit uses Poetry 2.X for package and dependency management.
+
+
+### Setup
+
+Clone the source repository with Git.
+
+```bash
+git clone git@github.com:IndicoDataSolutions/indico-toolkit-python.git
+```
+
+Install dependencies with Poetry.
+
+```bash
+poetry install
+```
+
+Formatting, linting, type checking, and tests are defined as
+[Poe](https://poethepoet.natn.io/) tasks in `pyproject.toml`.
+
+```bash
+poetry run poe {format,check,test,all}
+```
+
+Code changes or additions should pass `poetry run poe all` before opening a PR.
+
+
+### Tests
+
+Indico Toolkit has three test suites: required unit tests, extra unit tests, and
+integration tests.
+
+By default, only required unit tests are executed. Extra unit tests and integration
+tests are skipped.
+
+```bash
+poetry run poe {test,all}
+```
+
+Extra unit tests are skipped when their dependencies are not installed. To execute extra
+unit tests, install one or more extras and run the tests.
+
+```bash
+poetry install --all-extras
+poetry run poe {test,all}
+```
+
+Integration tests make API calls to an IPA environment and require a host and API token
+to execute. These tests create datasets, setup workflows, and train models. **Expect
+them to take tens of minutes to run.**
+
+```bash
+poetry run poe test-integration \
+    --host try.indico.io \
+    --token indico_api_token.txt
+```
+
+Make liberal use of pytest's `--last-failed` and `--failed-first`
+[flags](https://docs.pytest.org/en/stable/how-to/cache.html) to speed up integration
+test execution when writing code.
+

indico_toolkit-6.14.0/README.md

@@ -0,0 +1,109 @@
+# Indico Toolkit
+
+**This repository contains software that is not officially supported by Indico. It may
+  be outdated or contain bugs. The operations it performs are potentially destructive.
+  Use at your own risk.**
+
+Classes, functions, and abstractions for building workflows using the Indico IPA
+(Intelligent Process Automation) platform.
+
+- [Polling Classes](https://github.com/IndicoDataSolutions/indico-toolkit-python/tree/main/indico_toolkit/polling/__init__.py)
+  that implement best-practices polling behavior for Auto Review and Downstream
+  processes. Easily plug in business logic without the boilerplate.
+- [Result File](https://github.com/IndicoDataSolutions/indico-toolkit-python/blob/main/indico_toolkit/results/__init__.py)
+  and [Etl Output](https://github.com/IndicoDataSolutions/indico-toolkit-python/blob/main/indico_toolkit/etloutput/__init__.py)
+  Data Classes that parse standard IPA JSON output into idiomatic, type-safe Python dataclasses.
+- [Metrics Classes](https://github.com/IndicoDataSolutions/indico-toolkit-python/blob/main/indico_toolkit/metrics/__init__.py)
+  to compare model performance, evaluate ground truth, and plot statistics.
+- [Snapshot Classes](https://github.com/IndicoDataSolutions/indico-toolkit-python/blob/main/indico_toolkit/snapshots/snapshot.py)
+  to concatenate, merge, filter, and manipulate snapshot CSVs.
+
+...and more in the [Examples](https://github.com/IndicoDataSolutions/indico-toolkit-python/tree/main/examples) folder.
+
+
+## Installation
+
+**Indico Toolkit does not use semantic versioning.**
+
+Indico Toolkit versions match the minimum IPA version required to use its functionality.
+E.g. `indico-toolkit==6.14.0` makes use of functionality introduced in IPA 6.14, and
+some functionality requires IPA 6.14 or later to use.
+
+```bash
+pip install indico-toolkit
+```
+
+Some functionality requires optional dependencies that can be installed with extras.
+
+```bash
+pip install 'indico-toolkit[all]'
+pip install 'indico-toolkit[downloads]'
+pip install 'indico-toolkit[examples]'
+pip install 'indico-toolkit[metrics]'
+pip install 'indico-toolkit[predictions]'
+pip install 'indico-toolkit[snapshots]'
+```
+
+
+## Contributing
+
+Indico Toolkit uses Poetry 2.X for package and dependency management.
+
+
+### Setup
+
+Clone the source repository with Git.
+
+```bash
+git clone git@github.com:IndicoDataSolutions/indico-toolkit-python.git
+```
+
+Install dependencies with Poetry.
+
+```bash
+poetry install
+```
+
+Formatting, linting, type checking, and tests are defined as
+[Poe](https://poethepoet.natn.io/) tasks in `pyproject.toml`.
+
+```bash
+poetry run poe {format,check,test,all}
+```
+
+Code changes or additions should pass `poetry run poe all` before opening a PR.
+
+
+### Tests
+
+Indico Toolkit has three test suites: required unit tests, extra unit tests, and
+integration tests.
+
+By default, only required unit tests are executed. Extra unit tests and integration
+tests are skipped.
+
+```bash
+poetry run poe {test,all}
+```
+
+Extra unit tests are skipped when their dependencies are not installed. To execute extra
+unit tests, install one or more extras and run the tests.
+
+```bash
+poetry install --all-extras
+poetry run poe {test,all}
+```
+
+Integration tests make API calls to an IPA environment and require a host and API token
+to execute. These tests create datasets, setup workflows, and train models. **Expect
+them to take tens of minutes to run.**
+
+```bash
+poetry run poe test-integration \
+    --host try.indico.io \
+    --token indico_api_token.txt
+```
+
+Make liberal use of pytest's `--last-failed` and `--failed-first`
+[flags](https://docs.pytest.org/en/stable/how-to/cache.html) to speed up integration
+test execution when writing code.

indico_toolkit-6.14.0/indico_toolkit/__init__.py

@@ -0,0 +1,24 @@
+"""Classes, functions, and abstractions for Indico IPA"""
+
+from .client import create_client
+from .errors import (
+    ToolkitAuthError,
+    ToolkitError,
+    ToolkitInputError,
+    ToolkitInstantiationError,
+    ToolkitPopulationError,
+    ToolkitStaggeredLoopError,
+    ToolkitStatusError,
+)
+
+__all__ = (
+    "create_client",
+    "ToolkitAuthError",
+    "ToolkitError",
+    "ToolkitInputError",
+    "ToolkitInstantiationError",
+    "ToolkitPopulationError",
+    "ToolkitStaggeredLoopError",
+    "ToolkitStatusError",
+)
+__version__ = "6.14.0"

{indico_toolkit-6.1.0 → indico_toolkit-6.14.0}/indico_toolkit/association/__init__.py

@@ -3,3 +3,13 @@ from .extracted_tokens import ExtractedTokens
 from .line_items import LineItems
 from .positioning import Positioning
 from .splitting import split_prediction
+
+__all__ = (
+    "Association",
+    "ExtractedTokens",
+    "LineItems",
+    "Positioning",
+    "sequences_exact",
+    "sequences_overlap",
+    "split_prediction",
+)

{indico_toolkit-6.1.0 → indico_toolkit-6.14.0}/indico_toolkit/association/association.py

@@ -1,7 +1,8 @@
-from typing import Union, List, Dict
-from collections import defaultdict
 from abc import ABC, abstractmethod
-from indico_toolkit.types import Extractions
+from collections import defaultdict
+from typing import Dict, List, Union
+
+from ..types import Extractions
 
 
 class Association(ABC):

{indico_toolkit-6.1.0 → indico_toolkit-6.14.0}/indico_toolkit/association/extracted_tokens.py

@@ -1,29 +1,30 @@
 from typing import List, Union
-from copy import deepcopy
-from .association import sequences_overlap, Association, _check_if_token_match_found
-from indico_toolkit.types import Extractions
+
+from ..types import Extractions
+from .association import Association, _check_if_token_match_found, sequences_overlap
 
 # TODO: add test that 'errored_predictions' actually works with ValueErrors
 
 
 class ExtractedTokens(Association):
     """
-    Class to collect all extracted tokens, e.g. to enable highlighting of source document
+    Class to collect all extracted tokens, e.g. to enable highlighting of source
+    document
     """
 
-    def __init__(
-        self, predictions: Union[List[dict], Extractions],
-    ):
+    def __init__(self, predictions: Union[List[dict], Extractions]):
         super().__init__(predictions)
 
     def match_pred_to_token(self, pred: dict, ocr_tokens: List[dict], pred_index: int):
         """
-        Append matching token positions to self.mapped_positions, if no matches for pred, raise ValueError
+        Append matching token positions to self.mapped_positions, if no matches for
+        pred, raise ValueError
 
         Args:
             pred (dict): Indico extraction model prediction
             ocr_tokens (List[dict]): List of OCR tokens
-            pred_index (int): unique number for each prediction so that tokens can be linked to it
+            pred_index (int): unique number for each prediction so that tokens can be
+                linked to it
 
         Raises:
             ValueError: No matching token was found
@@ -46,21 +47,22 @@ class ExtractedTokens(Association):
         _check_if_token_match_found(pred, no_match)
         return match_token_index
 
-    def collect_tokens(
-        self, ocr_tokens: List[dict], raise_for_no_match: bool = True,
-    ):
+    def collect_tokens(self, ocr_tokens: List[dict], raise_for_no_match: bool = True):
         """
         Collect all extracted tokens and with pred text and label added to dictionaries
         Args:
         ocr_tokens (List[dict]): Tokens from 'ondocument' OCR config output
-        raise_for_no_match (bool): raise exception if a matching token isn't found for a prediction
+        raise_for_no_match (bool): raise exception if a matching token isn't found for a
+            prediction
         """
         self._separate_manually_added_predictions()
         self._predictions = self.sort_predictions_by_start_index(self._predictions)
         match_index = 0
         for pred_ind, pred in enumerate(self._predictions):
             try:
-                match_index = self.match_pred_to_token(pred, ocr_tokens[match_index:], pred_ind)
+                match_index = self.match_pred_to_token(
+                    pred, ocr_tokens[match_index:], pred_ind
+                )
             except ValueError as e:
                 if raise_for_no_match:
                     raise ValueError(e)

{indico_toolkit-6.1.0 → indico_toolkit-6.14.0}/indico_toolkit/association/line_items.py

@@ -1,14 +1,17 @@
 """Associate row items"""
-from typing import List, Union, Iterable, Dict
+
 from collections import defaultdict
 from copy import deepcopy
-from indico_toolkit.types import Extractions
-from .association import sequences_overlap, Association, _check_if_token_match_found
+from typing import Iterable, List, Union
+
+from ..types import Extractions
+from .association import Association, _check_if_token_match_found, sequences_overlap
 
 
 class LineItems(Association):
     """
-    Class for associating line items given extraction predictions and ondocument OCR tokens
+    Class for associating line items given extraction predictions and ondocument OCR
+    tokens
 
     Example Usage:
 
@@ -31,8 +34,8 @@ class LineItems(Association):
         """
         Args:
         predictions (List[dict]): List of extraction predictions
-        line_item_fields (Iterable[str]): Fields/labels to include as line item values, other values
-                                      will not be assigned a row_number.
+        line_item_fields (Iterable[str]): Fields/labels to include as line item values,
+            other values will not be assigned a row_number.
         """
         self.predictions = self.validate_prediction_formatting(predictions)
         self.line_item_fields: Iterable[str] = line_item_fields
@@ -78,13 +81,16 @@ class LineItems(Association):
         return match_token_index
 
     def get_bounding_boxes(
-        self, ocr_tokens: List[dict], raise_for_no_match: bool = True,
+        self, ocr_tokens: List[dict], raise_for_no_match: bool = True
     ):
         """
-        Adds keys for bounding box top/bottom/left/right and page number to line item predictions
+        Adds keys for bounding box top/bottom/left/right and page number to line item
+        predictions
+
         Args:
         ocr_tokens (List[dict]): Tokens from 'ondocument' OCR config output
-        raise_for_no_match (bool): raise exception if a matching token isn't found for a prediction
+        raise_for_no_match (bool): raise exception if a matching token isn't found for a
+            prediction
         """
         predictions = deepcopy(self.predictions)
         predictions = self._remove_unneeded_predictions(predictions)
@@ -117,7 +123,8 @@ class LineItems(Association):
         page_number = starting_pred["page_num"]
         row_number = 1
         for pred in self._mapped_positions:
-            # if the top of one box equals the bottom of another, we still want a new line
+            # if the top of one box equals the bottom of another, we still want a new
+            # line
             if pred["bbTop"] >= max_bot or pred["page_num"] != page_number:
                 row_number += 1
                 page_number = pred["page_num"]
@@ -129,8 +136,8 @@ class LineItems(Association):
     @property
     def grouped_line_items(self) -> List[List[dict]]:
         """
-        After row number has been assigned to predictions, returns line item predictions as a
-        list of lists where each list is a row.
+        After row number has been assigned to predictions, returns line item predictions
+        as a list of lists where each list is a row.
         """
         rows = defaultdict(list)
         for pred in self._mapped_positions:
@@ -139,7 +146,8 @@ class LineItems(Association):
 
     def _remove_unneeded_predictions(self, predictions: List[dict]) -> List[dict]:
         """
-        Remove predictions that are not line item fields or don't have valid start/end index data
+        Remove predictions that are not line item fields or don't have valid start/end
+        index data
         """
         valid_line_item_preds = []
         for pred in predictions:
@@ -159,8 +167,9 @@ class LineItems(Association):
 
     def _get_first_valid_line_item_pred(self) -> dict:
         if len(self._mapped_positions) == 0:
-            raise Exception(
-                "Whoops! You have no line_item_fields predictions. Did you run get_bounding_boxes?"
+            raise RuntimeError(
+                "Whoops! You have no line_item_fields predictions. "
+                "Did you run get_bounding_boxes?"
             )
         return self._mapped_positions[0]