hamtaa-texttools 1.1.2__py3-none-any.whl → 1.1.8__py3-none-any.whl

{hamtaa_texttools-1.1.2.dist-info → hamtaa_texttools-1.1.8.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hamtaa-texttools
-Version: 1.1.2
+Version: 1.1.8
 Summary: A high-level NLP toolkit built on top of modern LLMs.
 Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>
 License: MIT License
@@ -40,14 +40,14 @@ Dynamic: license-file
 
 It provides both **sync (`TheTool`)** and **async (`AsyncTheTool`)** APIs for maximum flexibility.
 
-It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extractor, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
+It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extraction, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
 ---
 
 ## ✨ Features
 
 TextTools provides a rich collection of high-level NLP utilities built on top of LLMs.  
-Each tool is designed to work out-of-the-box with structured outputs (JSON / Pydantic).
+Each tool is designed to work with structured outputs (JSON / Pydantic).
 
 - **`categorize()`** - Classifies text into Islamic studies categories 
 - **`is_question()`** - Binary detection of whether input is a question
@@ -63,7 +63,7 @@ Each tool is designed to work out-of-the-box with structured outputs (JSON / Pyd
 
 ---
 
-## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt` and `temperature` parameters
+## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature` and `validator` parameters
 
 TextTools provides several optional flags to customize LLM behavior:
 
@@ -78,12 +78,26 @@ Note: This doubles token usage per call because it triggers an additional LLM re
 
 - **`temperature=0.0`** → Determines how creative the model should respond. Takes a float number from `0.0` to `1.0`.
 
+- **`validator=validation_function`** → Forces TheTool to validate the output result based on your custom validator. Validator should return bool (True if there were no problem, False if the validation failed.) If validator failed, TheTool will retry to get another output by modifying `temperature`.
+
 All these parameters can be used individually or together to tailor the behavior of any tool in **TextTools**.
 
 **Note:** There might be some tools that don't support some of the parameters above.
 
 ---
 
+## 🧩 ToolOutput
+
+Every tool of `TextTools` returns a `ToolOutput` object which is a BaseModel with attributes:
+- **`result`** → The output of LLM (`type=Any`)
+- **`analysis`** → The reasoning step before generating the final output (`type=str`)
+- **`logprobs`** → Token-level probabilities for the generated output (`type=list`)
+- **`errors`** → Any error that have occured during calling LLM (`type=str`)
+
+**None:** You can use `repr(ToolOutput)` to see details of an output.
+
+---
+
 ## 🚀 Installation
 
 Install the latest release via PyPI:
@@ -121,13 +135,13 @@ the_tool = TheTool(client=client, model=model)
 detection = the_tool.is_question("Is this project open source?", logprobs=True, top_logprobs=2)
 print(detection.result)
 print(detection.logprobs)
-# Output: True \n --logprobs
+# Output: True + logprobs
 
 # Example: Translation
 translation = the_tool.translate("سلام، حالت چطوره؟" target_language="English", with_analysis=True)
 print(translation.result)
 print(translation.analysis)
-# Output: "Hi! How are you?" \n --analysis
+# Output: "Hi! How are you?"  + analysis
 ```
 
 ---
@@ -147,19 +161,22 @@ async def main():
     model = "gpt-4o-mini"
 
     # Create an instance of AsyncTheTool
-    the_tool = AsyncTheTool(client=async_client, model=model)
+    async_the_tool = AsyncTheTool(client=async_client, model=model)
+    
+    # Example: Async Translation and Keyword Extraction
+    translation_task = async_the_tool.translate("سلام، حالت چطوره؟", target_language="English")
+    keywords_task = async_the_tool.extract_keywords("Tomorrow, we will be dead by the car crash")
 
-    # Example: Async Translation
-    translation = await the_tool.translate("سلام، حالت چطوره؟", target_language="English")
+    (translation, keywords) = await asyncio.gather(translation_task, keywords_task)
     print(translation.result)
-    # Output: "Hi! How are you?"
+    print(keywords.result)
 
 asyncio.run(main())
 ```
 
 ---
 
-## 📚 Use Cases
+## 👍 Use Cases
 
 Use **TextTools** when you need to:
 
@@ -167,7 +184,35 @@ Use **TextTools** when you need to:
 - 🌍 **Translate** and process multilingual corpora with ease  
 - 🧩 **Integrate** LLMs into production pipelines (structured outputs)  
 - 📊 **Analyze** large text collections using embeddings and categorization  
-- 👍 **Automate** common text-processing tasks without reinventing the wheel  
+
+---
+
+## 📚 Batch Processing
+
+Process large datasets efficiently using OpenAI's batch API.
+
+## Quick Start
+
+```python
+from texttools import BatchJobRunner, BatchConfig
+
+# Configure your batch job
+config = BatchConfig(
+    system_prompt="Extract entities from the text",
+    job_name="entity_extraction",
+    input_data_path="data.json",
+    output_data_filename="results.json",
+    model="gpt-4o-mini"
+)
+
+# Define your output schema
+class Output(BaseModel):
+    entities: list[str]
+
+# Run the batch job
+runner = BatchJobRunner(config, output_model=Output)
+runner.run()
+```
 
 ---
 

hamtaa_texttools-1.1.8.dist-info/RECORD

@@ -0,0 +1,30 @@
+hamtaa_texttools-1.1.8.dist-info/licenses/LICENSE,sha256=Hb2YOBKy2MJQLnyLrX37B4ZVuac8eaIcE71SvVIMOLg,1082
+texttools/__init__.py,sha256=lFYe1jdssHC1h8qcPpV3whANxiDi8aiiFdY-7L0Ck10,164
+texttools/batch/__init__.py,sha256=DJGJTfR6F3Yv4_alsj9g1tesGzdcSV27Zw74DonhW_s,102
+texttools/batch/batch_manager.py,sha256=ZgLiO9maCHnx2cJbUjsYXFnlUsMLI2TP3Vc9uKU0BLg,8706
+texttools/batch/batch_runner.py,sha256=X0YQmaowO_jUSAFWBHdxOLoRrX_gvmrJDgp9qPlOSEw,10254
+texttools/prompts/README.md,sha256=-5YO93CN93QLifqZpUeUnCOCBbDiOTV-cFQeJ7Gg0I4,1377
+texttools/prompts/categorizer.yaml,sha256=GMqIIzQFhgnlpkgU1qi3FAD3mD4A2jiWD5TilQ2XnnE,1204
+texttools/prompts/extract_entities.yaml,sha256=KiKjeDpHaeh3JVtZ6q1pa3k4DYucUIU9WnEcRTCA-SE,651
+texttools/prompts/extract_keywords.yaml,sha256=0O7ypL_OsEOxtvlQ2CZjnsv9637DJwAKprZsf9Vo2_s,769
+texttools/prompts/is_question.yaml,sha256=d0-vKRbXWkxvO64ikvxRjEmpAXGpCYIPGhgexvPPjws,471
+texttools/prompts/merge_questions.yaml,sha256=0J85GvTirZB4ELwH3sk8ub_WcqqpYf6PrMKr3djlZeo,1792
+texttools/prompts/rewrite.yaml,sha256=LO7He_IA3MZKz8a-LxH9DHJpOjpYwaYN1pbjp1Y0tFo,5392
+texttools/prompts/run_custom.yaml,sha256=38OkCoVITbuuS9c08UZSP1jZW4WjSmRIi8fR0RAiPu4,108
+texttools/prompts/subject_to_question.yaml,sha256=C7x7rNNm6U_ZG9HOn6zuzYOtvJUZ2skuWbL1-aYdd3E,1147
+texttools/prompts/summarize.yaml,sha256=o6rxGPfWtZd61Duvm8NVvCJqfq73b-wAuMSKR6UYUqY,459
+texttools/prompts/text_to_question.yaml,sha256=UheKYpDn6iyKI8NxunHZtFpNyfCLZZe5cvkuXpurUJY,783
+texttools/prompts/translate.yaml,sha256=mGT2uBCei6uucWqVbs4silk-UV060v3G0jnt0P6sr50,634
+texttools/tools/__init__.py,sha256=3fPoeB-E5wGxWgv7axztHkeolR7ZDUJudd0xmpPFjao,113
+texttools/tools/async_tools.py,sha256=2ZY7Lo6Jj9xoTF8bfdh_g8VOXZ7ljMMesd1_QHXyf4s,15395
+texttools/tools/sync_tools.py,sha256=XKgZuzriFnk8B-YihJfs6BKivxjGCgOFfe7hnCpEiXs,15161
+texttools/tools/internals/async_operator.py,sha256=fCi70LXasC_2G9iz8uVFptnZEvVeb9TXopMBLi-cFuE,9022
+texttools/tools/internals/base_operator.py,sha256=rV2WqGdiHK4ezYz1f1EWcdbKFSFJhBJpORnJzPICFvk,3471
+texttools/tools/internals/formatters.py,sha256=tACNLP6PeoqaRpNudVxBaHA25zyWqWYPZQuYysIu88g,941
+texttools/tools/internals/operator.py,sha256=UBDScStTUXf8CIhwXb-6e_YOWTLggoiBV71vXRzr0P0,8904
+texttools/tools/internals/output_models.py,sha256=ekpbyocmXj_dee7ieOT1zOkMo9cPHT7xcUFCZoUaXA0,1886
+texttools/tools/internals/prompt_loader.py,sha256=1khayXcRC5w0Vf2SufpNaN1IUIhbKzS5ATiKheoBcGE,2082
+hamtaa_texttools-1.1.8.dist-info/METADATA,sha256=Cfb4VkcUELzRN6TrKdWK5jr4YsGbh_VlAtYVny86cb4,8690
+hamtaa_texttools-1.1.8.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+hamtaa_texttools-1.1.8.dist-info/top_level.txt,sha256=5Mh0jIxxZ5rOXHGJ6Mp-JPKviywwN0MYuH0xk5bEWqE,10
+hamtaa_texttools-1.1.8.dist-info/RECORD,,

texttools/__init__.py

@@ -1,9 +1,4 @@
-from .batch import BatchJobRunner, SimpleBatchManager
+from .batch import BatchJobRunner, BatchConfig
 from .tools import AsyncTheTool, TheTool
 
-__all__ = [
-    "TheTool",
-    "AsyncTheTool",
-    "SimpleBatchManager",
-    "BatchJobRunner",
-]
+__all__ = ["TheTool", "AsyncTheTool", "BatchJobRunner", "BatchConfig"]

texttools/batch/__init__.py

@@ -1,4 +1,3 @@
-from .batch_manager import SimpleBatchManager
-from .batch_runner import BatchJobRunner
+from .batch_runner import BatchJobRunner, BatchConfig
 
-__all__ = ["SimpleBatchManager", "BatchJobRunner"]
+__all__ = ["BatchJobRunner", "BatchConfig"]

texttools/batch/batch_manager.py

@@ -1,18 +1,20 @@
 import json
 import uuid
 from pathlib import Path
-from typing import Any, Type
+from typing import Any, Type, TypeVar
 import logging
 
 from pydantic import BaseModel
 from openai import OpenAI
 from openai.lib._pydantic import to_strict_json_schema
 
-logger = logging.getLogger("batch_runner")
-logger.setLevel(logging.INFO)
+# Base Model type for output models
+T = TypeVar("T", bound=BaseModel)
 
+logger = logging.getLogger("texttools.batch_manager")
 
-class SimpleBatchManager:
+
+class BatchManager:
     """
     Manages batch processing jobs for OpenAI's chat completions with structured outputs.
 
@@ -25,9 +27,8 @@ class SimpleBatchManager:
         self,
         client: OpenAI,
         model: str,
-        output_model: Type[BaseModel],
+        output_model: Type[T],
         prompt_template: str,
-        handlers: list[Any] | None = None,
         state_dir: Path = Path(".batch_jobs"),
         custom_json_schema_obj_str: dict | None = None,
         **client_kwargs: Any,
@@ -36,16 +37,16 @@ class SimpleBatchManager:
         self.model = model
         self.output_model = output_model
         self.prompt_template = prompt_template
-        self.handlers = handlers or []
         self.state_dir = state_dir
         self.state_dir.mkdir(parents=True, exist_ok=True)
         self.custom_json_schema_obj_str = custom_json_schema_obj_str
         self.client_kwargs = client_kwargs
         self.dict_input = False
 
-        if self.custom_json_schema_obj_str:
-            if self.custom_json_schema_obj_str is not dict:
-                raise ValueError("schema should be a dict")
+        if custom_json_schema_obj_str and not isinstance(
+            custom_json_schema_obj_str, dict
+        ):
+            raise ValueError("Schema should be a dict")
 
     def _state_file(self, job_name: str) -> Path:
         return self.state_dir / f"{job_name}.json"
@@ -126,7 +127,7 @@ class SimpleBatchManager:
 
         else:
             raise TypeError(
-                "The input must be either a list of texts or a dictionary in the form {'id': str, 'text': str}."
+                "The input must be either a list of texts or a dictionary in the form {'id': str, 'text': str}"
             )
 
         file_path = self.state_dir / f"batch_{uuid.uuid4().hex}.jsonl"
@@ -142,6 +143,7 @@ class SimpleBatchManager:
         """
         if self._load_state(job_name):
             return
+
         path = self._prepare_file(payload)
         upload = self.client.files.create(file=open(path, "rb"), purpose="batch")
         job = self.client.batches.create(
@@ -186,7 +188,7 @@ class SimpleBatchManager:
                 err_content = (
                     self.client.files.content(error_file_id).read().decode("utf-8")
                 )
-                logger.info("Error file content:", err_content)
+                logger.error("Error file content:", err_content)
             return {}
 
         content = self.client.files.content(out_file_id).read().decode("utf-8")
@@ -220,8 +222,6 @@ class SimpleBatchManager:
                 error_d = {custom_id: results[custom_id]}
                 log.append(error_d)
 
-        for handler in self.handlers:
-            handler.handle(results)
         if remove_cache:
             self._clear_state(job_name)
 

texttools/batch/batch_runner.py

@@ -3,24 +3,23 @@ import os
 import time
 from dataclasses import dataclass
 from pathlib import Path
-from typing import Any, Callable
+from typing import Any, Callable, Type, TypeVar
 import logging
 
 from dotenv import load_dotenv
 from openai import OpenAI
 from pydantic import BaseModel
 
-from texttools.batch import SimpleBatchManager
+from texttools.batch.batch_manager import BatchManager
+from texttools.tools.internals.output_models import StrOutput
 
-logger = logging.getLogger("batch_runner")
-logger.setLevel(logging.INFO)
+# Base Model type for output models
+T = TypeVar("T", bound=BaseModel)
 
+logger = logging.getLogger("texttools.batch_runner")
 
-class OutputModel(BaseModel):
-    desired_output: str
 
-
-def export_data(data):
+def export_data(data) -> list[dict[str, str]]:
     """
     Produces a structure of the following form from an initial data structure:
     [{"id": str, "text": str},...]
@@ -28,7 +27,7 @@ def export_data(data):
     return data
 
 
-def import_data(data):
+def import_data(data) -> Any:
     """
     Takes the output and adds and aggregates it to the original structure.
     """
@@ -47,9 +46,9 @@ class BatchConfig:
     output_data_filename: str = ""
     model: str = "gpt-4.1-mini"
     MAX_BATCH_SIZE: int = 100
-    MAX_TOTAL_TOKENS: int = 2000000
+    MAX_TOTAL_TOKENS: int = 2_000_000
     CHARS_PER_TOKEN: float = 2.7
-    PROMPT_TOKEN_MULTIPLIER: int = 1000
+    PROMPT_TOKEN_MULTIPLIER: int = 1_000
     BASE_OUTPUT_DIR: str = "Data/batch_entity_result"
     import_function: Callable = import_data
     export_function: Callable = export_data
@@ -63,7 +62,7 @@ class BatchJobRunner:
     """
 
     def __init__(
-        self, config: BatchConfig = BatchConfig(), output_model: type = OutputModel
+        self, config: BatchConfig = BatchConfig(), output_model: Type[T] = StrOutput
     ):
         self.config = config
         self.system_prompt = config.system_prompt
@@ -82,11 +81,11 @@ class BatchJobRunner:
         # Track retry attempts per part
         self.part_attempts: dict[int, int] = {}
 
-    def _init_manager(self) -> SimpleBatchManager:
+    def _init_manager(self) -> BatchManager:
         load_dotenv()
         api_key = os.getenv("OPENAI_API_KEY")
         client = OpenAI(api_key=api_key)
-        return SimpleBatchManager(
+        return BatchManager(
             client=client,
             model=self.model,
             prompt_template=self.system_prompt,
@@ -101,12 +100,12 @@ class BatchJobRunner:
         # Ensure data is a list of dicts with 'id' and 'content' as strings
         if not isinstance(data, list):
             raise ValueError(
-                'Exported data must be a list in this form:  [ {"id": str, "content": str},...]'
+                "Exported data must be a list of dicts with 'id' and 'content' keys"
             )
         for item in data:
             if not (isinstance(item, dict) and "id" in item and "content" in item):
                 raise ValueError(
-                    "Each item must be a dict with 'id' and 'content' keys."
+                    f"Item must be a dict with 'id' and 'content' keys. Got: {type(item)}"
                 )
             if not (isinstance(item["id"], str) and isinstance(item["content"], str)):
                 raise ValueError("'id' and 'content' must be strings.")
@@ -161,7 +160,45 @@ class BatchJobRunner:
             logger.info("Uploading...")
             time.sleep(30)
 
+    def _save_results(
+        self,
+        output_data: list[dict[str, Any]] | dict[str, Any],
+        log: list[Any],
+        part_idx: int,
+    ):
+        part_suffix = f"_part_{part_idx + 1}" if len(self.parts) > 1 else ""
+        result_path = (
+            Path(self.config.BASE_OUTPUT_DIR)
+            / f"{Path(self.output_data_filename).stem}{part_suffix}.json"
+        )
+        if not output_data:
+            logger.info("No output data to save. Skipping this part.")
+            return
+        else:
+            with open(result_path, "w", encoding="utf-8") as f:
+                json.dump(output_data, f, ensure_ascii=False, indent=4)
+        if log:
+            log_path = (
+                Path(self.config.BASE_OUTPUT_DIR)
+                / f"{Path(self.output_data_filename).stem}{part_suffix}_log.json"
+            )
+            with open(log_path, "w", encoding="utf-8") as f:
+                json.dump(log, f, ensure_ascii=False, indent=4)
+
+    def _result_exists(self, part_idx: int) -> bool:
+        part_suffix = f"_part_{part_idx + 1}" if len(self.parts) > 1 else ""
+        result_path = (
+            Path(self.config.BASE_OUTPUT_DIR)
+            / f"{Path(self.output_data_filename).stem}{part_suffix}.json"
+        )
+        return result_path.exists()
+
     def run(self):
+        """
+        Execute the batch job processing pipeline.
+
+        Submits jobs, monitors progress, handles retries, and saves results.
+        """
         # Submit all jobs up-front for concurrent execution
         self._submit_all_jobs()
         pending_parts: set[int] = set(self.part_idx_to_job_name.keys())
@@ -215,48 +252,3 @@ class BatchJobRunner:
                     f"Waiting {self.config.poll_interval_seconds}s before next status check for parts: {sorted(pending_parts)}"
                 )
                 time.sleep(self.config.poll_interval_seconds)
-
-    def _save_results(
-        self,
-        output_data: list[dict[str, Any]] | dict[str, Any],
-        log: list[Any],
-        part_idx: int,
-    ):
-        part_suffix = f"_part_{part_idx + 1}" if len(self.parts) > 1 else ""
-        result_path = (
-            Path(self.config.BASE_OUTPUT_DIR)
-            / f"{Path(self.output_data_filename).stem}{part_suffix}.json"
-        )
-        if not output_data:
-            logger.info("No output data to save. Skipping this part.")
-            return
-        else:
-            with open(result_path, "w", encoding="utf-8") as f:
-                json.dump(output_data, f, ensure_ascii=False, indent=4)
-        if log:
-            log_path = (
-                Path(self.config.BASE_OUTPUT_DIR)
-                / f"{Path(self.output_data_filename).stem}{part_suffix}_log.json"
-            )
-            with open(log_path, "w", encoding="utf-8") as f:
-                json.dump(log, f, ensure_ascii=False, indent=4)
-
-    def _result_exists(self, part_idx: int) -> bool:
-        part_suffix = f"_part_{part_idx + 1}" if len(self.parts) > 1 else ""
-        result_path = (
-            Path(self.config.BASE_OUTPUT_DIR)
-            / f"{Path(self.output_data_filename).stem}{part_suffix}.json"
-        )
-        return result_path.exists()
-
-
-if __name__ == "__main__":
-    logger.info("=== Batch Job Runner ===")
-    config = BatchConfig(
-        system_prompt="",
-        job_name="job_name",
-        input_data_path="Data.json",
-        output_data_filename="output",
-    )
-    runner = BatchJobRunner(config)
-    runner.run()

texttools/prompts/README.md

@@ -14,15 +14,15 @@ This folder contains YAML files for all prompts used in the project. Each file r
 ### Example YAML Structure
 ```yaml
 main_template:
-  default: |
+  mode_1: |
     Your main instructions here with placeholders like {input}.
-  reason: |
+  mode_2: |
     Optional reasoning instructions here.
 
 analyze_template:
-  default: |
+  mode_1: |
     Analyze and summarize the input.
-  reason: |
+  mode_2: |
     Optional detailed analysis template.
 ```
 

texttools/tools/__init__.py

@@ -1,4 +1,4 @@
-from .async_the_tool import AsyncTheTool
-from .the_tool import TheTool
+from .sync_tools import TheTool
+from .async_tools import AsyncTheTool
 
 __all__ = ["TheTool", "AsyncTheTool"]

texttools/tools/{async_the_tool.py → async_tools.py}

@@ -1,4 +1,4 @@
-from typing import Literal, Any
+from typing import Literal, Any, Callable
 
 from openai import AsyncOpenAI
 
@@ -34,6 +34,7 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Categorize a text into a single Islamic studies domain category.
@@ -52,6 +53,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="categorizer.yaml",
             output_model=OutputModels.CategorizerOutput,
@@ -69,6 +71,7 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Extract salient keywords from text.
@@ -88,6 +91,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="extract_keywords.yaml",
             output_model=OutputModels.ListStrOutput,
@@ -104,6 +108,7 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Perform Named Entity Recognition (NER) over the input text.
@@ -123,6 +128,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="extract_entities.yaml",
             output_model=OutputModels.ListDictStrStrOutput,
@@ -138,6 +144,7 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Detect if the input is phrased as a question.
@@ -156,6 +163,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="is_question.yaml",
             output_model=OutputModels.BoolOutput,
@@ -173,6 +181,7 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Generate a single question from the given text.
@@ -192,6 +201,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="text_to_question.yaml",
             output_model=OutputModels.StrOutput,
@@ -209,6 +219,7 @@ class AsyncTheTool:
         logprobs: bool = False,
         top_logprobs: int | None = None,
         mode: Literal["default", "reason"] = "default",
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Merge multiple questions into a single unified question.
@@ -229,6 +240,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="merge_questions.yaml",
             output_model=OutputModels.StrOutput,
@@ -246,6 +258,7 @@ class AsyncTheTool:
         logprobs: bool = False,
         top_logprobs: int | None = None,
         mode: Literal["positive", "negative", "hard_negative"] = "positive",
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Rewrite a text with different modes.
@@ -265,6 +278,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="rewrite.yaml",
             output_model=OutputModels.StrOutput,
@@ -282,6 +296,7 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Generate a list of questions about a subject.
@@ -302,6 +317,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="subject_to_question.yaml",
             output_model=OutputModels.ReasonListStrOutput,
@@ -318,6 +334,7 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Summarize the given subject text.
@@ -337,6 +354,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="summarize.yaml",
             output_model=OutputModels.StrOutput,
@@ -353,6 +371,7 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Translate text between languages.
@@ -372,6 +391,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="translate.yaml",
             output_model=OutputModels.StrOutput,
@@ -411,4 +431,5 @@ class AsyncTheTool:
             user_prompt=None,
             with_analysis=False,
             mode=None,
+            validator=None,
         )

texttools/tools/internals/async_operator.py

@@ -1,4 +1,4 @@
-from typing import Any, TypeVar, Type, Literal
+from typing import Any, TypeVar, Type, Literal, Callable
 import logging
 
 from openai import AsyncOpenAI
@@ -12,8 +12,7 @@ from texttools.tools.internals.prompt_loader import PromptLoader
 # Base Model type for output models
 T = TypeVar("T", bound=BaseModel)
 
-logger = logging.getLogger("async_operator")
-logger.setLevel(logging.INFO)
+logger = logging.getLogger("texttools.async_operator")
 
 
 class AsyncOperator(BaseOperator):
@@ -115,6 +114,7 @@ class AsyncOperator(BaseOperator):
         temperature: float,
         logprobs: bool,
         top_logprobs: int | None,
+        validator: Callable[[Any], bool] | None,
         # Internal parameters
         prompt_file: str,
         output_model: Type[T],
@@ -127,7 +127,7 @@ class AsyncOperator(BaseOperator):
         """
         prompt_loader = PromptLoader()
         formatter = Formatter()
-        output = ToolOutput(result="", analysis="", logprobs=[], errors=[])
+        output = ToolOutput()
 
         try:
             # Prompt configs contain two keys: main_template and analyze template, both are string
@@ -179,6 +179,55 @@ class AsyncOperator(BaseOperator):
 
             output.result = parsed.result
 
+            # Retry logic if validation fails
+            if validator and not validator(output.result):
+                max_retries = 3
+                for attempt in range(max_retries):
+                    logger.warning(
+                        f"Validation failed, retrying for the {attempt + 1} time."
+                    )
+
+                    # Generate new temperature for retry
+                    retry_temperature = self._get_retry_temp(temperature)
+                    try:
+                        if resp_format == "vllm":
+                            parsed, completion = await self._vllm_completion(
+                                messages,
+                                output_model,
+                                retry_temperature,
+                                logprobs,
+                                top_logprobs,
+                            )
+                        elif resp_format == "parse":
+                            parsed, completion = await self._parse_completion(
+                                messages,
+                                output_model,
+                                retry_temperature,
+                                logprobs,
+                                top_logprobs,
+                            )
+
+                        output.result = parsed.result
+
+                        # Check if retry was successful
+                        if validator(output.result):
+                            logger.info(
+                                f"Validation passed on retry attempt {attempt + 1}"
+                            )
+                            break
+                        else:
+                            logger.warning(
+                                f"Validation still failing after retry attempt {attempt + 1}"
+                            )
+
+                    except Exception as e:
+                        logger.error(f"Retry attempt {attempt + 1} failed: {e}")
+                        # Continue to next retry attempt if this one fails
+
+            # Final check after all retries
+            if validator and not validator(output.result):
+                output.errors.append("Validation failed after all retry attempts")
+
             if logprobs:
                 output.logprobs = self._extract_logprobs(completion)
 
@@ -189,4 +238,5 @@ class AsyncOperator(BaseOperator):
 
         except Exception as e:
             logger.error(f"AsyncTheTool failed: {e}")
-            return output.errors.append(str(e))
+            output.errors.append(str(e))
+            return output

texttools/tools/internals/base_operator.py

@@ -3,6 +3,7 @@ import json
 import re
 import math
 import logging
+import random
 
 from pydantic import BaseModel
 from openai import OpenAI, AsyncOpenAI
@@ -10,8 +11,7 @@ from openai import OpenAI, AsyncOpenAI
 # Base Model type for output models
 T = TypeVar("T", bound=BaseModel)
 
-logger = logging.getLogger("base_operator")
-logger.setLevel(logging.INFO)
+logger = logging.getLogger("texttools.base_operator")
 
 
 class BaseOperator:
@@ -86,3 +86,15 @@ class BaseOperator:
                 logprobs_data.append(token_entry)
 
         return logprobs_data
+
+    def _get_retry_temp(self, base_temp: float) -> float:
+        """
+        Calculate temperature for retry attempts.
+        """
+        delta_temp = random.choice([-1, 1]) * random.uniform(0.1, 0.9)
+        new_temp = base_temp + delta_temp
+        print(f"Base Temp: {base_temp}")
+        print(f"Delta Temp: {delta_temp}")
+        print(f"New Temp: {new_temp}")
+
+        return max(0.0, min(new_temp, 1.5))

texttools/tools/internals/operator.py

@@ -1,4 +1,4 @@
-from typing import Any, TypeVar, Type, Literal
+from typing import Any, TypeVar, Type, Literal, Callable
 import logging
 
 from openai import OpenAI
@@ -12,8 +12,7 @@ from texttools.tools.internals.prompt_loader import PromptLoader
 # Base Model type for output models
 T = TypeVar("T", bound=BaseModel)
 
-logger = logging.getLogger("operator")
-logger.setLevel(logging.INFO)
+logger = logging.getLogger("texttools.operator")
 
 
 class Operator(BaseOperator):
@@ -115,6 +114,7 @@ class Operator(BaseOperator):
         temperature: float,
         logprobs: bool,
         top_logprobs: int | None,
+        validator: Callable[[Any], bool] | None,
         # Internal parameters
         prompt_file: str,
         output_model: Type[T],
@@ -127,7 +127,7 @@ class Operator(BaseOperator):
         """
         prompt_loader = PromptLoader()
         formatter = Formatter()
-        output = ToolOutput(result="", analysis="", logprobs=[], errors=[])
+        output = ToolOutput()
 
         try:
             # Prompt configs contain two keys: main_template and analyze template, both are string
@@ -179,6 +179,55 @@ class Operator(BaseOperator):
 
             output.result = parsed.result
 
+            # Retry logic if validation fails
+            if validator and not validator(output.result):
+                max_retries = 3
+                for attempt in range(max_retries):
+                    logger.warning(
+                        f"Validation failed, retrying for the {attempt + 1} time."
+                    )
+
+                    # Generate new temperature for retry
+                    retry_temperature = self._get_retry_temp(temperature)
+                    try:
+                        if resp_format == "vllm":
+                            parsed, completion = self._vllm_completion(
+                                messages,
+                                output_model,
+                                retry_temperature,
+                                logprobs,
+                                top_logprobs,
+                            )
+                        elif resp_format == "parse":
+                            parsed, completion = self._parse_completion(
+                                messages,
+                                output_model,
+                                retry_temperature,
+                                logprobs,
+                                top_logprobs,
+                            )
+
+                        output.result = parsed.result
+
+                        # Check if retry was successful
+                        if validator(output.result):
+                            logger.info(
+                                f"Validation passed on retry attempt {attempt + 1}"
+                            )
+                            break
+                        else:
+                            logger.warning(
+                                f"Validation still failing after retry attempt {attempt + 1}"
+                            )
+
+                    except Exception as e:
+                        logger.error(f"Retry attempt {attempt + 1} failed: {e}")
+                        # Continue to next retry attempt if this one fails
+
+            # Final check after all retries
+            if validator and not validator(output.result):
+                output.errors.append("Validation failed after all retry attempts")
+
             if logprobs:
                 output.logprobs = self._extract_logprobs(completion)
 
@@ -188,5 +237,6 @@ class Operator(BaseOperator):
             return output
 
         except Exception as e:
-            logger.error(f"AsyncTheTool failed: {e}")
-            return output.errors.append(str(e))
+            logger.error(f"TheTool failed: {e}")
+            output.errors.append(str(e))
+            return output

texttools/tools/internals/output_models.py

@@ -4,10 +4,13 @@ from pydantic import BaseModel, Field
 
 
 class ToolOutput(BaseModel):
-    result: str
-    analysis: str
-    logprobs: list[dict[str, Any]]
-    errors: list[str]
+    result: Any = None
+    analysis: str = ""
+    logprobs: list[dict[str, Any]] = []
+    errors: list[str] = []
+
+    def __repr__(self) -> str:
+        return f"ToolOutput(result_type='{type(self.result)}', result='{self.result}', analysis='{self.analysis}', logprobs='{self.logprobs}', errors='{self.errors}'"
 
 
 class StrOutput(BaseModel):

texttools/tools/{the_tool.py → sync_tools.py}

@@ -1,4 +1,4 @@
-from typing import Literal, Any
+from typing import Literal, Any, Callable
 
 from openai import OpenAI
 
@@ -32,6 +32,7 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Categorize a text into a single Islamic studies domain category.
@@ -50,6 +51,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="categorizer.yaml",
             output_model=OutputModels.CategorizerOutput,
@@ -67,6 +69,7 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Extract salient keywords from text.
@@ -86,6 +89,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="extract_keywords.yaml",
             output_model=OutputModels.ListStrOutput,
@@ -102,6 +106,7 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Perform Named Entity Recognition (NER) over the input text.
@@ -121,6 +126,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="extract_entities.yaml",
             output_model=OutputModels.ListDictStrStrOutput,
@@ -136,6 +142,7 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Detect if the input is phrased as a question.
@@ -154,6 +161,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="is_question.yaml",
             output_model=OutputModels.BoolOutput,
@@ -171,6 +179,7 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Generate a single question from the given text.
@@ -190,6 +199,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="text_to_question.yaml",
             output_model=OutputModels.StrOutput,
@@ -207,6 +217,7 @@ class TheTool:
         logprobs: bool = False,
         top_logprobs: int | None = None,
         mode: Literal["default", "reason"] = "default",
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Merge multiple questions into a single unified question.
@@ -227,6 +238,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="merge_questions.yaml",
             output_model=OutputModels.StrOutput,
@@ -244,6 +256,7 @@ class TheTool:
         logprobs: bool = False,
         top_logprobs: int | None = None,
         mode: Literal["positive", "negative", "hard_negative"] = "positive",
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Rewrite a text with different modes.
@@ -263,6 +276,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="rewrite.yaml",
             output_model=OutputModels.StrOutput,
@@ -280,6 +294,7 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Generate a list of questions about a subject.
@@ -300,6 +315,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="subject_to_question.yaml",
             output_model=OutputModels.ReasonListStrOutput,
@@ -316,6 +332,7 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Summarize the given subject text.
@@ -335,6 +352,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="summarize.yaml",
             output_model=OutputModels.StrOutput,
@@ -351,6 +369,7 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
+        validator: Callable[[Any], bool] | None = None,
     ) -> OutputModels.ToolOutput:
         """
         Translate text between languages.
@@ -370,6 +389,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="translate.yaml",
             output_model=OutputModels.StrOutput,
@@ -409,4 +429,5 @@ class TheTool:
             user_prompt=None,
             with_analysis=False,
             mode=None,
+            validator=None,
         )

hamtaa_texttools-1.1.2.dist-info/RECORD

@@ -1,30 +0,0 @@
-hamtaa_texttools-1.1.2.dist-info/licenses/LICENSE,sha256=Hb2YOBKy2MJQLnyLrX37B4ZVuac8eaIcE71SvVIMOLg,1082
-texttools/__init__.py,sha256=v3tQCH_Cjj47fCpuhK6sKSVAqEjNkc-cZbY4OJa4IZw,202
-texttools/batch/__init__.py,sha256=q50JsQsmQGp_8RW0KNasYeYWVV0R4FUNZ-ujXwEJemY,143
-texttools/batch/batch_manager.py,sha256=UCJaOq-tTy6kTZJvFuBDBSmzlDeVlLTeFlR83e6eXkQ,8808
-texttools/batch/batch_runner.py,sha256=JNlwKkCdSDGOxp7Wwl4F2MaXMcmXK9DG0oaNOqUKNpI,10293
-texttools/prompts/README.md,sha256=rclMaCV1N8gT1KcpZu0-ka0dKGNg2f1CEcRMdQkgQOc,1379
-texttools/prompts/categorizer.yaml,sha256=GMqIIzQFhgnlpkgU1qi3FAD3mD4A2jiWD5TilQ2XnnE,1204
-texttools/prompts/extract_entities.yaml,sha256=KiKjeDpHaeh3JVtZ6q1pa3k4DYucUIU9WnEcRTCA-SE,651
-texttools/prompts/extract_keywords.yaml,sha256=0O7ypL_OsEOxtvlQ2CZjnsv9637DJwAKprZsf9Vo2_s,769
-texttools/prompts/is_question.yaml,sha256=d0-vKRbXWkxvO64ikvxRjEmpAXGpCYIPGhgexvPPjws,471
-texttools/prompts/merge_questions.yaml,sha256=0J85GvTirZB4ELwH3sk8ub_WcqqpYf6PrMKr3djlZeo,1792
-texttools/prompts/rewrite.yaml,sha256=LO7He_IA3MZKz8a-LxH9DHJpOjpYwaYN1pbjp1Y0tFo,5392
-texttools/prompts/run_custom.yaml,sha256=38OkCoVITbuuS9c08UZSP1jZW4WjSmRIi8fR0RAiPu4,108
-texttools/prompts/subject_to_question.yaml,sha256=C7x7rNNm6U_ZG9HOn6zuzYOtvJUZ2skuWbL1-aYdd3E,1147
-texttools/prompts/summarize.yaml,sha256=o6rxGPfWtZd61Duvm8NVvCJqfq73b-wAuMSKR6UYUqY,459
-texttools/prompts/text_to_question.yaml,sha256=UheKYpDn6iyKI8NxunHZtFpNyfCLZZe5cvkuXpurUJY,783
-texttools/prompts/translate.yaml,sha256=mGT2uBCei6uucWqVbs4silk-UV060v3G0jnt0P6sr50,634
-texttools/tools/__init__.py,sha256=hG1I28Q7BJ1Dbs95x6QMKXdsAlC5Eh_tqC-EbAibwiU,114
-texttools/tools/async_the_tool.py,sha256=ORn3xk6zJ3HdDNo_zj9cTlY05zsQ1un5H96xLIUUOg4,14446
-texttools/tools/the_tool.py,sha256=q-bvYv38CuBMstYOV_42zv2frHgsaLRzo_XAGNrdNWA,14212
-texttools/tools/internals/async_operator.py,sha256=vZvU5XPdNKz--ZYheEQOIRzVNi-Ni1x9GxgcDBlbYJM,6760
-texttools/tools/internals/base_operator.py,sha256=3GUSEN87Y_1CqkeK_QXkFG8O7GPliGSVUtiPp1taQRM,3051
-texttools/tools/internals/formatters.py,sha256=tACNLP6PeoqaRpNudVxBaHA25zyWqWYPZQuYysIu88g,941
-texttools/tools/internals/operator.py,sha256=M-0-6SmRmW7PX4CWhMAgoSwtk_2w1_YrItK3FhdObCU,6659
-texttools/tools/internals/output_models.py,sha256=gbVbzBWeyHUVNsCBuawdgz9ZEzsC7wfygGgZJsAaexY,1662
-texttools/tools/internals/prompt_loader.py,sha256=1khayXcRC5w0Vf2SufpNaN1IUIhbKzS5ATiKheoBcGE,2082
-hamtaa_texttools-1.1.2.dist-info/METADATA,sha256=J2YfhGNN2SpOq5P3iOsniaUzpuUaP20dTlPBiQX6Gss,7144
-hamtaa_texttools-1.1.2.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-hamtaa_texttools-1.1.2.dist-info/top_level.txt,sha256=5Mh0jIxxZ5rOXHGJ6Mp-JPKviywwN0MYuH0xk5bEWqE,10
-hamtaa_texttools-1.1.2.dist-info/RECORD,,