hamtaa-texttools 1.1.0__tar.gz → 1.1.5__tar.gz

{hamtaa_texttools-1.1.0/hamtaa_texttools.egg-info → hamtaa_texttools-1.1.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hamtaa-texttools
-Version: 1.1.0
+Version: 1.1.5
 Summary: A high-level NLP toolkit built on top of modern LLMs.
 Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>
 License: MIT License
@@ -29,7 +29,7 @@ Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: openai==1.97.1
-Requires-Dist: PyYAML>=6.0
+Requires-Dist: pyyaml>=6.0
 Dynamic: license-file
 
 # TextTools
@@ -40,14 +40,14 @@ Dynamic: license-file
 
 It provides both **sync (`TheTool`)** and **async (`AsyncTheTool`)** APIs for maximum flexibility.
 
-It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extractor, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
+It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extraction, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
 ---
 
 ## ✨ Features
 
 TextTools provides a rich collection of high-level NLP utilities built on top of LLMs.  
-Each tool is designed to work out-of-the-box with structured outputs (JSON / Pydantic).
+Each tool is designed to work with structured outputs (JSON / Pydantic).
 
 - **`categorize()`** - Classifies text into Islamic studies categories 
 - **`is_question()`** - Binary detection of whether input is a question
@@ -63,7 +63,7 @@ Each tool is designed to work out-of-the-box with structured outputs (JSON / Pyd
 
 ---
 
-## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt` and `temperature` parameters
+## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature` and `validator` parameters
 
 TextTools provides several optional flags to customize LLM behavior:
 
@@ -78,6 +78,8 @@ Note: This doubles token usage per call because it triggers an additional LLM re
 
 - **`temperature=0.0`** → Determines how creative the model should respond. Takes a float number from `0.0` to `1.0`.
 
+- **`validator=validation_function`** → Forces TheTool to validate the output result based on your custom validator. Validator should return bool (True if there were no problem, False if the validation failed.) If validator failed, TheTool will retry to get another output by modifying `temperature`.
+
 All these parameters can be used individually or together to tailor the behavior of any tool in **TextTools**.
 
 **Note:** There might be some tools that don't support some of the parameters above.

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/README.md

@@ -6,14 +6,14 @@
 
 It provides both **sync (`TheTool`)** and **async (`AsyncTheTool`)** APIs for maximum flexibility.
 
-It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extractor, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
+It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extraction, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
 ---
 
 ## ✨ Features
 
 TextTools provides a rich collection of high-level NLP utilities built on top of LLMs.  
-Each tool is designed to work out-of-the-box with structured outputs (JSON / Pydantic).
+Each tool is designed to work with structured outputs (JSON / Pydantic).
 
 - **`categorize()`** - Classifies text into Islamic studies categories 
 - **`is_question()`** - Binary detection of whether input is a question
@@ -29,7 +29,7 @@ Each tool is designed to work out-of-the-box with structured outputs (JSON / Pyd
 
 ---
 
-## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt` and `temperature` parameters
+## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature` and `validator` parameters
 
 TextTools provides several optional flags to customize LLM behavior:
 
@@ -44,6 +44,8 @@ Note: This doubles token usage per call because it triggers an additional LLM re
 
 - **`temperature=0.0`** → Determines how creative the model should respond. Takes a float number from `0.0` to `1.0`.
 
+- **`validator=validation_function`** → Forces TheTool to validate the output result based on your custom validator. Validator should return bool (True if there were no problem, False if the validation failed.) If validator failed, TheTool will retry to get another output by modifying `temperature`.
+
 All these parameters can be used individually or together to tailor the behavior of any tool in **TextTools**.
 
 **Note:** There might be some tools that don't support some of the parameters above.

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5/hamtaa_texttools.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hamtaa-texttools
-Version: 1.1.0
+Version: 1.1.5
 Summary: A high-level NLP toolkit built on top of modern LLMs.
 Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>
 License: MIT License
@@ -29,7 +29,7 @@ Requires-Python: >=3.8
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: openai==1.97.1
-Requires-Dist: PyYAML>=6.0
+Requires-Dist: pyyaml>=6.0
 Dynamic: license-file
 
 # TextTools
@@ -40,14 +40,14 @@ Dynamic: license-file
 
 It provides both **sync (`TheTool`)** and **async (`AsyncTheTool`)** APIs for maximum flexibility.
 
-It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extractor, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
+It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extraction, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
 ---
 
 ## ✨ Features
 
 TextTools provides a rich collection of high-level NLP utilities built on top of LLMs.  
-Each tool is designed to work out-of-the-box with structured outputs (JSON / Pydantic).
+Each tool is designed to work with structured outputs (JSON / Pydantic).
 
 - **`categorize()`** - Classifies text into Islamic studies categories 
 - **`is_question()`** - Binary detection of whether input is a question
@@ -63,7 +63,7 @@ Each tool is designed to work out-of-the-box with structured outputs (JSON / Pyd
 
 ---
 
-## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt` and `temperature` parameters
+## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature` and `validator` parameters
 
 TextTools provides several optional flags to customize LLM behavior:
 
@@ -78,6 +78,8 @@ Note: This doubles token usage per call because it triggers an additional LLM re
 
 - **`temperature=0.0`** → Determines how creative the model should respond. Takes a float number from `0.0` to `1.0`.
 
+- **`validator=validation_function`** → Forces TheTool to validate the output result based on your custom validator. Validator should return bool (True if there were no problem, False if the validation failed.) If validator failed, TheTool will retry to get another output by modifying `temperature`.
+
 All these parameters can be used individually or together to tailor the behavior of any tool in **TextTools**.
 
 **Note:** There might be some tools that don't support some of the parameters above.

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/hamtaa_texttools.egg-info/requires.txt

@@ -1,2 +1,2 @@
 openai==1.97.1
-PyYAML>=6.0
+pyyaml>=6.0

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "hamtaa-texttools"
-version = "1.1.0"
+version = "1.1.5"
 authors = [
   { name = "Tohidi", email = "the.mohammad.tohidi@gmail.com" },
   { name = "Montazer", email = "montazerh82@gmail.com" },
@@ -17,7 +17,7 @@ license = {file = "LICENSE"}
 requires-python = ">=3.8"
 dependencies = [
   "openai==1.97.1",
-  "PyYAML>=6.0",
+  "pyyaml>=6.0",
 ]
 keywords = ["nlp", "llm", "text-processing", "openai"]
 

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/batch/batch_manager.py

@@ -8,7 +8,6 @@ from pydantic import BaseModel
 from openai import OpenAI
 from openai.lib._pydantic import to_strict_json_schema
 
-# Configure logger
 logger = logging.getLogger("batch_runner")
 logger.setLevel(logging.INFO)
 

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/batch/batch_runner.py

@@ -12,7 +12,6 @@ from pydantic import BaseModel
 
 from texttools.batch import SimpleBatchManager
 
-# Configure logger
 logger = logging.getLogger("batch_runner")
 logger.setLevel(logging.INFO)
 

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/tools/async_the_tool.py

@@ -1,4 +1,4 @@
-from typing import Literal, Any
+from typing import Literal, Any, Callable
 
 from openai import AsyncOpenAI
 
@@ -34,12 +34,16 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Categorize a text into a single Islamic studies domain category.
 
         Returns:
-            {"result": <category string>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The assigned Islamic studies category
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -49,6 +53,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="categorizer.yaml",
             output_model=OutputModels.CategorizerOutput,
@@ -66,12 +71,16 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, list[str]]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Extract salient keywords from text.
 
         Returns:
-            {"result": [<keyword1>, <keyword2>, ...]} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (list[str]): List of extracted keywords
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -82,6 +91,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="extract_keywords.yaml",
             output_model=OutputModels.ListStrOutput,
@@ -98,12 +108,16 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, list[dict[str, str]]]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Perform Named Entity Recognition (NER) over the input text.
 
         Returns:
-            {"result": [{"text": <entity>, "type": <entity_type>}, ...]} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (list[dict]): List of entities with 'text' and 'type' keys
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -114,6 +128,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="extract_entities.yaml",
             output_model=OutputModels.ListDictStrStrOutput,
@@ -129,12 +144,16 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, bool]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Detect if the input is phrased as a question.
 
         Returns:
-            {"result": True} or {"result": False} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (bool): True if text is a question, False otherwise
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -144,6 +163,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="is_question.yaml",
             output_model=OutputModels.BoolOutput,
@@ -161,12 +181,16 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Generate a single question from the given text.
 
         Returns:
-            {"result": <generated_question>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The generated question
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -177,6 +201,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="text_to_question.yaml",
             output_model=OutputModels.StrOutput,
@@ -194,12 +219,16 @@ class AsyncTheTool:
         logprobs: bool = False,
         top_logprobs: int | None = None,
         mode: Literal["default", "reason"] = "default",
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Merge multiple questions into a single unified question.
 
         Returns:
-            {"result": <merged_question>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The merged question
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         text = ", ".join(text)
         return await self.operator.run(
@@ -211,6 +240,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="merge_questions.yaml",
             output_model=OutputModels.StrOutput,
@@ -228,12 +258,16 @@ class AsyncTheTool:
         logprobs: bool = False,
         top_logprobs: int | None = None,
         mode: Literal["positive", "negative", "hard_negative"] = "positive",
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Rewrite a text with different modes.
 
         Returns:
-            {"result": <rewritten_text>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The rewritten text
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -244,6 +278,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="rewrite.yaml",
             output_model=OutputModels.StrOutput,
@@ -261,12 +296,16 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, list[str]]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Generate a list of questions about a subject.
 
         Returns:
-            {"result": [<question1>, <question2>, ...]} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (list[str]): List of generated questions
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -278,6 +317,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="subject_to_question.yaml",
             output_model=OutputModels.ReasonListStrOutput,
@@ -294,12 +334,16 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Summarize the given subject text.
 
         Returns:
-            {"result": <summary>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The summary text
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -310,6 +354,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="summarize.yaml",
             output_model=OutputModels.StrOutput,
@@ -326,12 +371,16 @@ class AsyncTheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Translate text between languages.
 
         Returns:
-            {"result": <translated_text>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The translated text
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return await self.operator.run(
             # User parameters
@@ -342,6 +391,7 @@ class AsyncTheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="translate.yaml",
             output_model=OutputModels.StrOutput,
@@ -358,12 +408,13 @@ class AsyncTheTool:
         temperature: float | None = None,
         logprobs: bool | None = None,
         top_logprobs: int | None = None,
-    ) -> dict[str, Any]:
+    ) -> OutputModels.ToolOutput:
         """
         Custom tool that can do almost anything!
 
         Returns:
-            {"result": <Any>}
+            ToolOutput: Object with fields:
+                - result (str): The output result
         """
         return await self.operator.run(
             # User paramaeters
@@ -380,4 +431,5 @@ class AsyncTheTool:
             user_prompt=None,
             with_analysis=False,
             mode=None,
+            validator=None,
         )

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/tools/internals/async_operator.py

@@ -1,4 +1,4 @@
-from typing import Any, TypeVar, Type, Literal
+from typing import Any, TypeVar, Type, Literal, Callable
 import logging
 
 from openai import AsyncOpenAI
@@ -12,7 +12,6 @@ from texttools.tools.internals.prompt_loader import PromptLoader
 # Base Model type for output models
 T = TypeVar("T", bound=BaseModel)
 
-# Configure logger
 logger = logging.getLogger("async_operator")
 logger.setLevel(logging.INFO)
 
@@ -32,6 +31,10 @@ class AsyncOperator(BaseOperator):
         self.model = model
 
     async def _analyze(self, prompt_configs: dict[str, str], temperature: float) -> str:
+        """
+        Calls OpenAI API for analysis using the configured prompt template.
+        Returns the analyzed content as a string.
+        """
         analyze_prompt = prompt_configs["analyze_template"]
         analyze_message = [self._build_user_message(analyze_prompt)]
         completion = await self.client.chat.completions.create(
@@ -50,6 +53,10 @@ class AsyncOperator(BaseOperator):
         logprobs: bool = False,
         top_logprobs: int = 3,
     ) -> tuple[Type[T], Any]:
+        """
+        Parses a chat completion using OpenAI's structured output format.
+        Returns both the parsed object and the raw completion for logging.
+        """
         request_kwargs = {
             "model": self.model,
             "messages": message,
@@ -73,6 +80,10 @@ class AsyncOperator(BaseOperator):
         logprobs: bool = False,
         top_logprobs: int = 3,
     ) -> tuple[Type[T], Any]:
+        """
+        Generates a completion using vLLM with JSON schema guidance.
+        Returns the parsed output model and raw completion.
+        """
         json_schema = output_model.model_json_schema()
 
         # Build kwargs dynamically
@@ -104,20 +115,23 @@ class AsyncOperator(BaseOperator):
         temperature: float,
         logprobs: bool,
         top_logprobs: int | None,
+        validator: Callable[[Any], bool] | None,
         # Internal parameters
         prompt_file: str,
         output_model: Type[T],
         resp_format: Literal["vllm", "parse"],
         mode: str | None,
         **extra_kwargs,
-    ) -> dict[str, Any]:
+    ) -> ToolOutput:
         """
         Execute the async LLM pipeline with the given input text. (Async)
         """
         prompt_loader = PromptLoader()
         formatter = Formatter()
+        output = ToolOutput()
 
         try:
+            # Prompt configs contain two keys: main_template and analyze template, both are string
             prompt_configs = prompt_loader.load(
                 prompt_file=prompt_file,
                 text=text.strip(),
@@ -159,14 +173,62 @@ class AsyncOperator(BaseOperator):
 
             # Ensure output_model has a `result` field
             if not hasattr(parsed, "result"):
-                logger.error(
-                    "The provided output_model must define a field named 'result'"
-                )
-
-            output = ToolOutput(result="", analysis="", logprobs=[], errors=[])
+                error = "The provided output_model must define a field named 'result'"
+                logger.error(error)
+                output.errors.append(error)
+                return output
 
             output.result = parsed.result
 
+            # Retry logic if validation fails
+            if validator and not validator(output.result):
+                max_retries = 3
+                for attempt in range(max_retries):
+                    logger.warning(
+                        f"Validation failed, retrying for the {attempt + 1} time."
+                    )
+
+                    # Generate new temperature for retry
+                    retry_temperature = self._get_retry_temp(temperature)
+                    try:
+                        if resp_format == "vllm":
+                            parsed, completion = await self._vllm_completion(
+                                messages,
+                                output_model,
+                                retry_temperature,
+                                logprobs,
+                                top_logprobs,
+                            )
+                        elif resp_format == "parse":
+                            parsed, completion = await self._parse_completion(
+                                messages,
+                                output_model,
+                                retry_temperature,
+                                logprobs,
+                                top_logprobs,
+                            )
+
+                        output.result = parsed.result
+
+                        # Check if retry was successful
+                        if validator(output.result):
+                            logger.info(
+                                f"Validation passed on retry attempt {attempt + 1}"
+                            )
+                            break
+                        else:
+                            logger.warning(
+                                f"Validation still failing after retry attempt {attempt + 1}"
+                            )
+
+                    except Exception as e:
+                        logger.error(f"Retry attempt {attempt + 1} failed: {e}")
+                        # Continue to next retry attempt if this one fails
+
+            # Final check after all retries
+            if validator and not validator(output.result):
+                output.errors.append("Validation failed after all retry attempts")
+
             if logprobs:
                 output.logprobs = self._extract_logprobs(completion)
 
@@ -174,6 +236,7 @@ class AsyncOperator(BaseOperator):
                 output.analysis = analysis
 
             return output
+
         except Exception as e:
             logger.error(f"AsyncTheTool failed: {e}")
-            return ToolOutput(result="", analysis="", logprobs=[], errors=[str(e)])
+            return output.errors.append(str(e))

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/tools/internals/base_operator.py

@@ -3,6 +3,7 @@ import json
 import re
 import math
 import logging
+import random
 
 from pydantic import BaseModel
 from openai import OpenAI, AsyncOpenAI
@@ -10,7 +11,6 @@ from openai import OpenAI, AsyncOpenAI
 # Base Model type for output models
 T = TypeVar("T", bound=BaseModel)
 
-# Configure logger
 logger = logging.getLogger("base_operator")
 logger.setLevel(logging.INFO)
 
@@ -40,13 +40,6 @@ class BaseOperator:
     ) -> Type[T]:
         """
         Convert a JSON response string to output model.
-
-        Args:
-            response_string: The JSON string (may contain code block markers)
-            output_model: Your Pydantic output model class (e.g., StrOutput, ListStrOutput)
-
-        Returns:
-            Instance of your output model
         """
         # Clean the response string
         cleaned_json = self._clean_json_response(response_string)
@@ -61,7 +54,12 @@ class BaseOperator:
         return output_model(**response_dict)
 
     def _extract_logprobs(self, completion: dict) -> list[dict[str, Any]]:
+        """
+        Extracts and filters token probabilities from completion logprobs.
+        Skips punctuation and structural tokens, returns cleaned probability data.
+        """
         logprobs_data = []
+
         ignore_pattern = re.compile(r'^(result|[\s\[\]\{\}",:]+)$')
 
         for choice in completion.choices:
@@ -89,3 +87,15 @@ class BaseOperator:
                 logprobs_data.append(token_entry)
 
         return logprobs_data
+
+    def _get_retry_temp(self, base_temp: float) -> float:
+        """
+        Calculate temperature for retry attempts.
+        """
+        delta_temp = random.choice([-1, 1]) * random.uniform(0.1, 0.9)
+        new_temp = base_temp + delta_temp
+        print(f"Base Temp: {base_temp}")
+        print(f"Delta Temp: {delta_temp}")
+        print(f"New Temp: {new_temp}")
+
+        return max(0.0, min(new_temp, 1.5))

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/tools/internals/operator.py

@@ -1,4 +1,4 @@
-from typing import Any, TypeVar, Type, Literal
+from typing import Any, TypeVar, Type, Literal, Callable
 import logging
 
 from openai import OpenAI
@@ -12,7 +12,6 @@ from texttools.tools.internals.prompt_loader import PromptLoader
 # Base Model type for output models
 T = TypeVar("T", bound=BaseModel)
 
-# Configure logger
 logger = logging.getLogger("operator")
 logger.setLevel(logging.INFO)
 
@@ -32,6 +31,10 @@ class Operator(BaseOperator):
         self.model = model
 
     def _analyze(self, prompt_configs: dict[str, str], temperature: float) -> str:
+        """
+        Calls OpenAI API for analysis using the configured prompt template.
+        Returns the analyzed content as a string.
+        """
         analyze_prompt = prompt_configs["analyze_template"]
         analyze_message = [self._build_user_message(analyze_prompt)]
         completion = self.client.chat.completions.create(
@@ -50,6 +53,10 @@ class Operator(BaseOperator):
         logprobs: bool = False,
         top_logprobs: int = 3,
     ) -> tuple[Type[T], Any]:
+        """
+        Parses a chat completion using OpenAI's structured output format.
+        Returns both the parsed object and the raw completion for logging.
+        """
         request_kwargs = {
             "model": self.model,
             "messages": message,
@@ -73,6 +80,10 @@ class Operator(BaseOperator):
         logprobs: bool = False,
         top_logprobs: int = 3,
     ) -> tuple[Type[T], Any]:
+        """
+        Generates a completion using vLLM with JSON schema guidance.
+        Returns the parsed output model and raw completion.
+        """
         json_schema = output_model.model_json_schema()
 
         # Build kwargs dynamically
@@ -104,20 +115,23 @@ class Operator(BaseOperator):
         temperature: float,
         logprobs: bool,
         top_logprobs: int | None,
+        validator: Callable[[Any], bool] | None,
         # Internal parameters
         prompt_file: str,
         output_model: Type[T],
         resp_format: Literal["vllm", "parse"],
         mode: str | None,
         **extra_kwargs,
-    ) -> dict[str, Any]:
+    ) -> ToolOutput:
         """
         Execute the LLM pipeline with the given input text.
         """
         prompt_loader = PromptLoader()
         formatter = Formatter()
+        output = ToolOutput()
 
         try:
+            # Prompt configs contain two keys: main_template and analyze template, both are string
             prompt_configs = prompt_loader.load(
                 prompt_file=prompt_file,
                 text=text.strip(),
@@ -159,14 +173,62 @@ class Operator(BaseOperator):
 
             # Ensure output_model has a `result` field
             if not hasattr(parsed, "result"):
-                logger.error(
-                    "The provided output_model must define a field named 'result'"
-                )
-
-            output = ToolOutput(result="", analysis="", logprobs=[], errors=[])
+                error = "The provided output_model must define a field named 'result'"
+                logger.error(error)
+                output.errors.append(error)
+                return output
 
             output.result = parsed.result
 
+            # Retry logic if validation fails
+            if validator and not validator(output.result):
+                max_retries = 3
+                for attempt in range(max_retries):
+                    logger.warning(
+                        f"Validation failed, retrying for the {attempt + 1} time."
+                    )
+
+                    # Generate new temperature for retry
+                    retry_temperature = self._get_retry_temp(temperature)
+                    try:
+                        if resp_format == "vllm":
+                            parsed, completion = self._vllm_completion(
+                                messages,
+                                output_model,
+                                retry_temperature,
+                                logprobs,
+                                top_logprobs,
+                            )
+                        elif resp_format == "parse":
+                            parsed, completion = self._parse_completion(
+                                messages,
+                                output_model,
+                                retry_temperature,
+                                logprobs,
+                                top_logprobs,
+                            )
+
+                        output.result = parsed.result
+
+                        # Check if retry was successful
+                        if validator(output.result):
+                            logger.info(
+                                f"Validation passed on retry attempt {attempt + 1}"
+                            )
+                            break
+                        else:
+                            logger.warning(
+                                f"Validation still failing after retry attempt {attempt + 1}"
+                            )
+
+                    except Exception as e:
+                        logger.error(f"Retry attempt {attempt + 1} failed: {e}")
+                        # Continue to next retry attempt if this one fails
+
+            # Final check after all retries
+            if validator and not validator(output.result):
+                output.errors.append("Validation failed after all retry attempts")
+
             if logprobs:
                 output.logprobs = self._extract_logprobs(completion)
 
@@ -174,6 +236,7 @@ class Operator(BaseOperator):
                 output.analysis = analysis
 
             return output
+
         except Exception as e:
             logger.error(f"TheTool failed: {e}")
-            return ToolOutput(result="", analysis="", logprobs=[], errors=[str(e)])
+            return output.errors.append(str(e))

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/tools/internals/output_models.py

@@ -4,10 +4,13 @@ from pydantic import BaseModel, Field
 
 
 class ToolOutput(BaseModel):
-    result: str
-    analysis: str
-    logprobs: list[dict[str, Any]]
-    errors: list[str]
+    result: Any = None
+    analysis: str = ""
+    logprobs: list[dict[str, Any]] = []
+    errors: list[str] = []
+
+    def __repr__(self) -> str:
+        return f"ToolOutput(result_type='{type(self.result)}', result='{self.result}', analysis='{self.analysis}', logprobs='{self.logprobs}', errors='{self.errors}'"
 
 
 class StrOutput(BaseModel):

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/tools/internals/prompt_loader.py

@@ -24,6 +24,9 @@ class PromptLoader:
     # Use lru_cache to load each file once
     @lru_cache(maxsize=32)
     def _load_templates(self, prompt_file: str, mode: str | None) -> dict[str, str]:
+        """
+        Loads prompt templates from YAML file with optional mode selection.
+        """
         base_dir = Path(__file__).parent.parent.parent / Path("prompts")
         prompt_path = base_dir / prompt_file
         data = yaml.safe_load(prompt_path.read_text(encoding="utf-8"))

{hamtaa_texttools-1.1.0 → hamtaa_texttools-1.1.5}/texttools/tools/the_tool.py

@@ -1,4 +1,4 @@
-from typing import Literal, Any
+from typing import Literal, Any, Callable
 
 from openai import OpenAI
 
@@ -32,12 +32,16 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Categorize a text into a single Islamic studies domain category.
 
         Returns:
-            {"result": <category string>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The assigned Islamic studies category
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -47,6 +51,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="categorizer.yaml",
             output_model=OutputModels.CategorizerOutput,
@@ -64,12 +69,16 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, list[str]]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Extract salient keywords from text.
 
         Returns:
-            {"result": [<keyword1>, <keyword2>, ...]} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (list[str]): List of extracted keywords
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -80,6 +89,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="extract_keywords.yaml",
             output_model=OutputModels.ListStrOutput,
@@ -96,12 +106,16 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, list[dict[str, str]]]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Perform Named Entity Recognition (NER) over the input text.
 
         Returns:
-            {"result": [{"text": <entity>, "type": <entity_type>}, ...]} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (list[dict]): List of entities with 'text' and 'type' keys
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -112,6 +126,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="extract_entities.yaml",
             output_model=OutputModels.ListDictStrStrOutput,
@@ -127,12 +142,16 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, bool]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Detect if the input is phrased as a question.
 
         Returns:
-            {"result": True} or {"result": False} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (bool): True if text is a question, False otherwise
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -142,6 +161,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="is_question.yaml",
             output_model=OutputModels.BoolOutput,
@@ -159,12 +179,16 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Generate a single question from the given text.
 
         Returns:
-            {"result": <generated_question>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The generated question
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -175,6 +199,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="text_to_question.yaml",
             output_model=OutputModels.StrOutput,
@@ -192,12 +217,16 @@ class TheTool:
         logprobs: bool = False,
         top_logprobs: int | None = None,
         mode: Literal["default", "reason"] = "default",
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Merge multiple questions into a single unified question.
 
         Returns:
-            {"result": <merged_question>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The merged question
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         text = ", ".join(text)
         return self.operator.run(
@@ -209,6 +238,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="merge_questions.yaml",
             output_model=OutputModels.StrOutput,
@@ -226,12 +256,16 @@ class TheTool:
         logprobs: bool = False,
         top_logprobs: int | None = None,
         mode: Literal["positive", "negative", "hard_negative"] = "positive",
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Rewrite a text with different modes.
 
         Returns:
-            {"result": <rewritten_text>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The rewritten text
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -242,6 +276,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="rewrite.yaml",
             output_model=OutputModels.StrOutput,
@@ -259,12 +294,16 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, list[str]]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Generate a list of questions about a subject.
 
         Returns:
-            {"result": [<question1>, <question2>, ...]} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (list[str]): List of generated questions
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -276,6 +315,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="subject_to_question.yaml",
             output_model=OutputModels.ReasonListStrOutput,
@@ -292,12 +332,16 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Summarize the given subject text.
 
         Returns:
-            {"result": <summary>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The summary text
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -308,6 +352,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="summarize.yaml",
             output_model=OutputModels.StrOutput,
@@ -324,12 +369,16 @@ class TheTool:
         temperature: float | None = 0.0,
         logprobs: bool = False,
         top_logprobs: int | None = None,
-    ) -> dict[str, str]:
+        validator: Callable[[Any], bool] | None = None,
+    ) -> OutputModels.ToolOutput:
         """
         Translate text between languages.
 
         Returns:
-            {"result": <translated_text>} + ("logprobs" and "analysis" if enabled)
+            ToolOutput: Object containing:
+                - result (str): The translated text
+                - logprobs (list | None): Probability data if logprobs enabled
+                - analysis (str | None): Detailed reasoning if with_analysis enabled
         """
         return self.operator.run(
             # User parameters
@@ -340,6 +389,7 @@ class TheTool:
             temperature=temperature,
             logprobs=logprobs,
             top_logprobs=top_logprobs,
+            validator=validator,
             # Internal parameters
             prompt_file="translate.yaml",
             output_model=OutputModels.StrOutput,
@@ -356,12 +406,13 @@ class TheTool:
         temperature: float | None = None,
         logprobs: bool | None = None,
         top_logprobs: int | None = None,
-    ) -> dict[str, Any]:
+    ) -> OutputModels.ToolOutput:
         """
         Custom tool that can do almost anything!
 
         Returns:
-            {"result": <Any>}
+            ToolOutput: Object with fields:
+                - result (str): The output result
         """
         return self.operator.run(
             # User paramaeters
@@ -378,4 +429,5 @@ class TheTool:
             user_prompt=None,
             with_analysis=False,
             mode=None,
+            validator=None,
         )