hamtaa-texttools 1.0.2__tar.gz → 1.0.3__tar.gz

{hamtaa_texttools-1.0.2/hamtaa_texttools.egg-info → hamtaa_texttools-1.0.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hamtaa-texttools
-Version: 1.0.2
+Version: 1.0.3
 Summary: TextTools is a high-level NLP toolkit built on top of modern LLMs.
 Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>
 License: MIT License
@@ -37,8 +37,11 @@ Dynamic: license-file
 ## 📌 Overview
 
 **TextTools** is a high-level **NLP toolkit** built on top of modern **LLMs**.  
+
 It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extractor, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
+**Thread Safety:** All methods in TheTool are thread-safe, allowing concurrent usage across multiple threads without conflicts.
+
 ---
 
 ## ✨ Features
@@ -59,11 +62,20 @@ Each tool is designed to work out-of-the-box with structured outputs (JSON / Pyd
 
 ---
 
-## 🔍 `with_analysis` Mode
+## ⚙️ with_analysis, logprobs, output_lang, and user_prompt parameters
+
+TextTools provides several optional flags to customize LLM behavior:
+
+- **`with_analysis=True`** → Adds a reasoning step before generating the final output. Useful for debugging, improving prompts, or understanding model behavior.  
+  ⚠️ Note: This doubles token usage per call because it triggers an additional LLM request.
+
+- **`logprobs=True`** → Returns token-level probabilities for the generated output. You can also specify `top_logprobs=<N>` to get the top N alternative tokens and their probabilities.  
+
+- **`output_lang="en"`** → Forces the model to respond in a specific language. The model will ignore other instructions about language and respond strictly in the requested language.
 
-The `with_analysis=True` flag enhances the tool's output by providing a detailed reasoning chain behind its result. This is valuable for debugging, improving prompts, or understanding model behavior.
+- **`user_prompt="..."`** → Allows you to inject a custom instruction or prompt into the model alongside the main template. This gives you fine-grained control over how the model interprets or modifies the input text.
 
-**Please be aware:** This feature works by making an additional LLM API call for each tool invocation, which will **effectively double your token usage** for that operation.
+All these flags can be used individually or together to tailor the behavior of any tool in **TextTools**.
 
 ---
 
@@ -99,7 +111,7 @@ print(the_tool.detect_question("Is this project open source?")["result"])
 # Output: True
 
 # Example: Translation
-print(the_tool.translate("سلام، حالت چطوره؟")["result"])
+print(the_tool.translate("سلام، حالت چطوره؟", target_language="English")["result"])
 # Output: "Hi! How are you?"
 ```
 
@@ -113,7 +125,7 @@ Use **TextTools** when you need to:
 - 🌍 **Translate** and process multilingual corpora with ease  
 - 🧩 **Integrate** LLMs into production pipelines (structured outputs)  
 - 📊 **Analyze** large text collections using embeddings and categorization  
-- ⚙️ **Automate** common text-processing tasks without reinventing the wheel  
+- 👍 **Automate** common text-processing tasks without reinventing the wheel  
 
 ---
 

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/README.md

@@ -3,8 +3,11 @@
 ## 📌 Overview
 
 **TextTools** is a high-level **NLP toolkit** built on top of modern **LLMs**.  
+
 It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extractor, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
+**Thread Safety:** All methods in TheTool are thread-safe, allowing concurrent usage across multiple threads without conflicts.
+
 ---
 
 ## ✨ Features
@@ -25,11 +28,20 @@ Each tool is designed to work out-of-the-box with structured outputs (JSON / Pyd
 
 ---
 
-## 🔍 `with_analysis` Mode
+## ⚙️ with_analysis, logprobs, output_lang, and user_prompt parameters
+
+TextTools provides several optional flags to customize LLM behavior:
+
+- **`with_analysis=True`** → Adds a reasoning step before generating the final output. Useful for debugging, improving prompts, or understanding model behavior.  
+  ⚠️ Note: This doubles token usage per call because it triggers an additional LLM request.
+
+- **`logprobs=True`** → Returns token-level probabilities for the generated output. You can also specify `top_logprobs=<N>` to get the top N alternative tokens and their probabilities.  
+
+- **`output_lang="en"`** → Forces the model to respond in a specific language. The model will ignore other instructions about language and respond strictly in the requested language.
 
-The `with_analysis=True` flag enhances the tool's output by providing a detailed reasoning chain behind its result. This is valuable for debugging, improving prompts, or understanding model behavior.
+- **`user_prompt="..."`** → Allows you to inject a custom instruction or prompt into the model alongside the main template. This gives you fine-grained control over how the model interprets or modifies the input text.
 
-**Please be aware:** This feature works by making an additional LLM API call for each tool invocation, which will **effectively double your token usage** for that operation.
+All these flags can be used individually or together to tailor the behavior of any tool in **TextTools**.
 
 ---
 
@@ -65,7 +77,7 @@ print(the_tool.detect_question("Is this project open source?")["result"])
 # Output: True
 
 # Example: Translation
-print(the_tool.translate("سلام، حالت چطوره؟")["result"])
+print(the_tool.translate("سلام، حالت چطوره؟", target_language="English")["result"])
 # Output: "Hi! How are you?"
 ```
 
@@ -79,7 +91,7 @@ Use **TextTools** when you need to:
 - 🌍 **Translate** and process multilingual corpora with ease  
 - 🧩 **Integrate** LLMs into production pipelines (structured outputs)  
 - 📊 **Analyze** large text collections using embeddings and categorization  
-- ⚙️ **Automate** common text-processing tasks without reinventing the wheel  
+- 👍 **Automate** common text-processing tasks without reinventing the wheel  
 
 ---
 

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3/hamtaa_texttools.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hamtaa-texttools
-Version: 1.0.2
+Version: 1.0.3
 Summary: TextTools is a high-level NLP toolkit built on top of modern LLMs.
 Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>
 License: MIT License
@@ -37,8 +37,11 @@ Dynamic: license-file
 ## 📌 Overview
 
 **TextTools** is a high-level **NLP toolkit** built on top of modern **LLMs**.  
+
 It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extractor, and more** — designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
+**Thread Safety:** All methods in TheTool are thread-safe, allowing concurrent usage across multiple threads without conflicts.
+
 ---
 
 ## ✨ Features
@@ -59,11 +62,20 @@ Each tool is designed to work out-of-the-box with structured outputs (JSON / Pyd
 
 ---
 
-## 🔍 `with_analysis` Mode
+## ⚙️ with_analysis, logprobs, output_lang, and user_prompt parameters
+
+TextTools provides several optional flags to customize LLM behavior:
+
+- **`with_analysis=True`** → Adds a reasoning step before generating the final output. Useful for debugging, improving prompts, or understanding model behavior.  
+  ⚠️ Note: This doubles token usage per call because it triggers an additional LLM request.
+
+- **`logprobs=True`** → Returns token-level probabilities for the generated output. You can also specify `top_logprobs=<N>` to get the top N alternative tokens and their probabilities.  
+
+- **`output_lang="en"`** → Forces the model to respond in a specific language. The model will ignore other instructions about language and respond strictly in the requested language.
 
-The `with_analysis=True` flag enhances the tool's output by providing a detailed reasoning chain behind its result. This is valuable for debugging, improving prompts, or understanding model behavior.
+- **`user_prompt="..."`** → Allows you to inject a custom instruction or prompt into the model alongside the main template. This gives you fine-grained control over how the model interprets or modifies the input text.
 
-**Please be aware:** This feature works by making an additional LLM API call for each tool invocation, which will **effectively double your token usage** for that operation.
+All these flags can be used individually or together to tailor the behavior of any tool in **TextTools**.
 
 ---
 
@@ -99,7 +111,7 @@ print(the_tool.detect_question("Is this project open source?")["result"])
 # Output: True
 
 # Example: Translation
-print(the_tool.translate("سلام، حالت چطوره؟")["result"])
+print(the_tool.translate("سلام، حالت چطوره؟", target_language="English")["result"])
 # Output: "Hi! How are you?"
 ```
 
@@ -113,7 +125,7 @@ Use **TextTools** when you need to:
 - 🌍 **Translate** and process multilingual corpora with ease  
 - 🧩 **Integrate** LLMs into production pipelines (structured outputs)  
 - 📊 **Analyze** large text collections using embeddings and categorization  
-- ⚙️ **Automate** common text-processing tasks without reinventing the wheel  
+- 👍 **Automate** common text-processing tasks without reinventing the wheel  
 
 ---
 

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/hamtaa_texttools.egg-info/SOURCES.txt

@@ -7,8 +7,10 @@ hamtaa_texttools.egg-info/SOURCES.txt
 hamtaa_texttools.egg-info/dependency_links.txt
 hamtaa_texttools.egg-info/requires.txt
 hamtaa_texttools.egg-info/top_level.txt
-tests/test_tools.py
 texttools/__init__.py
+texttools/batch/__init__.py
+texttools/batch/batch_manager.py
+texttools/batch/batch_runner.py
 texttools/formatters/base_formatter.py
 texttools/formatters/user_merge_formatter/user_merge_formatter.py
 texttools/prompts/README.md
@@ -23,11 +25,9 @@ texttools/prompts/subject_question_generator.yaml
 texttools/prompts/summarizer.yaml
 texttools/prompts/translator.yaml
 texttools/tools/__init__.py
-texttools/tools/operator.py
-texttools/tools/output_models.py
-texttools/tools/prompt_loader.py
+texttools/tools/async_the_tool.py
 texttools/tools/the_tool.py
-texttools/utils/__init__.py
-texttools/utils/batch_manager/__init__.py
-texttools/utils/batch_manager/batch_manager.py
-texttools/utils/batch_manager/batch_runner.py
+texttools/tools/internals/async_operator.py
+texttools/tools/internals/operator.py
+texttools/tools/internals/output_models.py
+texttools/tools/internals/prompt_loader.py

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "hamtaa-texttools"
-version = "1.0.2"
+version = "1.0.3"
 authors = [
   { name = "Tohidi", email = "the.mohammad.tohidi@gmail.com" },
   { name = "Montazer", email = "montazerh82@gmail.com" },

hamtaa_texttools-1.0.3/texttools/__init__.py

@@ -0,0 +1,9 @@
+from .batch import BatchJobRunner, SimpleBatchManager
+from .tools import AsyncTheTool, TheTool
+
+__all__ = [
+    "TheTool",
+    "AsyncTheTool",
+    "SimpleBatchManager",
+    "BatchJobRunner",
+]

{hamtaa_texttools-1.0.2/texttools/utils/batch_manager → hamtaa_texttools-1.0.3/texttools/batch}/batch_runner.py

@@ -8,7 +8,7 @@ from typing import Any, Callable
 from openai import OpenAI
 from pydantic import BaseModel
 
-from texttools.utils.batch_manager import SimpleBatchManager
+from texttools.batch.batch_manager import SimpleBatchManager
 
 
 class Output(BaseModel):

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/formatters/user_merge_formatter/user_merge_formatter.py

@@ -13,24 +13,7 @@ class UserMergeFormatter(BaseFormatter):
         ValueError: If the input messages have invalid structure or roles.
     """
 
-    def _validate_input(self, messages: list[dict[str, str]]):
-        valid_keys = {"role", "content"}
-        valid_roles = {"user", "assistant"}
-
-        for message in messages:
-            # Validate keys
-            if set(message.keys()) != valid_keys:
-                raise ValueError(
-                    f"Message dict keys must be exactly {valid_keys}, got {set(message.keys())}"
-                )
-            # Validate roles
-            role = message["role"]
-            if role != "system" and role not in valid_roles:
-                raise ValueError(f"Unexpected role: {role}")
-
     def format(self, messages: list[dict[str, str]]) -> list[dict[str, str]]:
-        self._validate_input(messages)
-
         merged: list[dict[str, str]] = []
 
         for message in messages:

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/README.md

@@ -7,20 +7,20 @@ This folder contains YAML files for all prompts used in the project. Each file r
 - **prompt_file.yaml**: Each YAML file represents a single prompt template.
 - **main_template**: The main instruction template for the model.
 - **analyze_template** (optional): A secondary reasoning template used before generating the final response.
-- **Modes** (optional): Some prompts may have multiple modes (e.g., `default_mode`, `reason_mode`) to allow different behaviors.
+- **Modes** (optional): Some prompts may have multiple modes (e.g., `default`, `reason`) to allow different behaviors.
 
 ### Example YAML Structure
 ```yaml
 main_template:
-  default_mode: |
+  default: |
     Your main instructions here with placeholders like {input}.
-  reason_mode: |
+  reason: |
     Optional reasoning instructions here.
 
 analyze_template:
-  default_mode: |
+  default: |
     Analyze and summarize the input.
-  reason_mode: |
+  reason: |
     Optional detailed analysis template.
 ```
 

hamtaa_texttools-1.0.3/texttools/prompts/categorizer.yaml

@@ -0,0 +1,31 @@
+main_template: |
+  {user_prompt}
+  تو یک متخصص علوم دینی هستی
+  من به عنوان کاربر یک متن به تو میدم و از تو میخوام که
+  اون متن رو در یکی از دسته بندی های زیر طبقه بندی کنی
+
+  "باورهای دینی",
+  "اخلاق اسلامی",
+  "احکام و فقه",
+  "تاریخ اسلام و شخصیت ها",
+  "منابع دینی",
+  "دین و جامعه/سیاست",
+  "عرفان و معنویت",
+  "هیچکدام",
+
+  فقط با این فرمت json پاسخ بده:
+  {{
+  	  "reason": "<دلیل انتخابت رو به صورت خلاصه بگو>",
+      "result": "<یکی از دسته بندی ها>"
+  }}
+
+  متنی که باید طبقه بندی کنی:
+  {input}
+
+analyze_template: |
+  هدف ما طبقه بندی متن هست
+  متن رو بخون و ایده اصلی و آنالیزی کوتاه از اون رو ارائه بده
+
+  بسیار خلاصه باشه خروجی تو
+  نهایتا 20 کلمه
+  {input}

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/keyword_extractor.yaml

@@ -1,9 +1,12 @@
 main_template: |
+  {user_prompt}
   Extract the most relevant keywords from the following text. Provide them as a list of strings.
-  {input}
   Respond only in JSON format:
   {{"result": ["keyword1", "keyword2", ...]}}.
+  No addition, No comments, No explanation.
   Respond in the language of the input text.
+  Here is the text:
+  {input}
 
 analyze_template: |
   Analyze the following text to identify its main topics, concepts, and important terms.

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/ner_extractor.yaml

@@ -1,7 +1,7 @@
 main_template: |
+  {user_prompt}
   Identify and extract all named entities (e.g., PER, ORG, LOC, DAT, etc.) from the following text. For each entity, provide its text and a clear type.
   Respond as a JSON array of objects.
-  {input}
   Respond only in JSON format:
   {{
     "result": [
@@ -11,6 +11,9 @@ main_template: |
       }}
     ]
   }}
+  No addition, No comments, No explanation.
+  Here is the text:
+  {input}
 
 analyze_template: |
   Read the following text and identify any proper nouns, key concepts, or specific mentions that might represent named entities.

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/question_detector.yaml

@@ -1,7 +1,10 @@
 main_template: |
+  {user_prompt}
   Determine that if the following text contains any question or request of some kind or not.
-  Respond only in JSON format:
-  {{"result": "true/false"}}
+  Respond only in JSON format (Output should be a bool):
+  {{"result": True/False}}
+  No addition, No comments, No explanation.
+  Here is the text:
   {input}
 
 analyze_template: |

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/question_generator.yaml

@@ -1,15 +1,16 @@
 main_template: |
+  {user_prompt}
   Given the following answer, generate a single, 
   appropriate question that this answer would directly respond to.
   The generated answer should be independently meaningful,
   and not mentioning any verbs like, this, that, he or she on the question.
   The generated question must be in the language of the user's input.
-  Here is the text:
-  {input}
   Respond only in JSON format:
   {{"result": "string"}}
   Respond only with the new generated question, without any additional information.          
-
+  Here is the text:
+  {input}
+  
 analyze_template: |
   Analyze the following answer to identify its key facts,
   main subject, and what kind of information it provides.

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/question_merger.yaml

@@ -1,6 +1,7 @@
 main_template:
 
-  default_mode: |
+  default: |
+    {user_prompt}
     You are a language expert.
     I will give you a list of questions that are semantically similar.
     Your task is to merge them into one unified question that:
@@ -15,7 +16,8 @@ main_template:
     Respond only with the new generated question, without any additional information.
     The generated question must be in the language of the users input.
 
-  reason_mode: |
+  reason: |
+    {user_prompt}
     You are an AI assistant helping to unify semantically similar questions.
     First, briefly extract the unique intent or content from each input question.
     Then, write one merged question that combines all their content clearly and naturally, without redundancy.
@@ -29,7 +31,7 @@ main_template:
 
 analyze_template:
 
-  default_mode: | 
+  default: | 
     Analyze the following questions to identify their core intent, key concepts, 
     and the specific information they are seeking.
     Provide a brief, summarized understanding of the questions' meaning that
@@ -37,7 +39,7 @@ analyze_template:
     Respond in the language of the question.
     Here is the question: {input}
 
-  reason_mode: |
+  reason: |
     Analyze the following questions to identify their exact wording, phrasing,
     and the literal meaning it conveys.
     Provide a brief, summarized analysis of their linguistic structure and current meaning,

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/question_rewriter.yaml

@@ -1,6 +1,7 @@
 main_template:
 
-  same_meaning_different_wording_mode: |
+  same_meaning_different_wording: |
+    {user_prompt}
     Rewrite the following question using completely different wording and phrasing, 
     ensuring its original meaning is perfectly preserved. The rewritten question 
     should be distinct from the original but convey the exact same inquiry.
@@ -11,7 +12,8 @@ main_template:
     Respond only with the new generated question, without any additional information.
     The generated question must be in the language of the users input.
 
-  different_meaning_similar_wording_mode: |
+  different_meaning_similar_wording: |
+    {user_prompt}
     Rewrite the following question using *very similar wording and phrasing* 
     to the original, but ensure the rewritten question has a *completely different meaning*. 
     Focus on subtle changes that drastically alter the intent or subject of the question.
@@ -24,7 +26,7 @@ main_template:
 
 analyze_template:
 
-  same_meaning_different_wording_mode: | 
+  same_meaning_different_wording: | 
     Analyze the following question to identify its core intent, key concepts, 
     and the specific information it is seeking.
     Provide a brief, summarized understanding of the question's meaning that
@@ -33,7 +35,7 @@ analyze_template:
     Here is the question:
     {input}
 
-  different_meaning_similar_wording_mode: |
+  different_meaning_similar_wording: |
     Analyze the following question to identify its exact wording, phrasing,
     and the literal meaning it conveys.
     Provide a brief, summarized analysis of its linguistic structure and current meaning,

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/subject_question_generator.yaml

@@ -1,19 +1,19 @@
 main_template: |
+  {user_prompt}
   Given the following subject, generate {number_of_questions} appropriate questions that this subject would directly respond to.
   The generated subject should be independently meaningful,
   and it must not mention any verbs like, this, that, he or she and etc. in the question.
-  The generated question must be in {language} language.
   Here is the text:
   {input}
   Respond only with the new generated question, without any additional information.
-  The generated question must be in {language} language.
   Generate {number_of_questions} number of questions in the questions list.
   You must return ONLY a single JSON object that matches the schema.
   There is a `reason` key, fill that up with a really summerized version
   of your thoughts.
   The `reason` must be less than 20 words.
+  Don't forget to fill the reason.
   Respond only in JSON format:
-  {{"result": [string, string, ...]}}
+  {{"result": [str, str, ...], "reason": str}}
 
 analyze_template: |
   Our goal is to generate questions, from the given subject that I've provided.
@@ -24,4 +24,3 @@ analyze_template: |
   What point of views can we see and generate questoins from it? (Questions that real users might have.)
   Here is the subject:
   {input}
-  Respond only in {language} language.

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/summarizer.yaml

@@ -1,4 +1,5 @@
 main_template: |
+  {user_prompt}
   Provide a concise summary of the following text:
   {input}
   Respond only in JSON format:

{hamtaa_texttools-1.0.2 → hamtaa_texttools-1.0.3}/texttools/prompts/translator.yaml

@@ -1,4 +1,5 @@
 main_template: |
+  {user_prompt}
   You are a {target_language} translator.
   Output only the translated text. No comments, no explanations, no markdown.
   Translate the following text to {target_language}:

hamtaa_texttools-1.0.3/texttools/tools/__init__.py

@@ -0,0 +1,4 @@
+from .async_the_tool import AsyncTheTool
+from .the_tool import TheTool
+
+__all__ = ["TheTool", "AsyncTheTool"]