hamtaa-texttools 1.1.13__py3-none-any.whl → 1.1.14__py3-none-any.whl

{hamtaa_texttools-1.1.13.dist-info → hamtaa_texttools-1.1.14.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hamtaa-texttools
-Version: 1.1.13
+Version: 1.1.14
 Summary: A high-level NLP toolkit built on top of modern LLMs.
 Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>
 License: MIT License
@@ -50,7 +50,7 @@ It provides ready-to-use utilities for **translation, question detection, keywor
 TextTools provides a rich collection of high-level NLP utilities,
 Each tool is designed to work with structured outputs (JSON / Pydantic).
 
-- **`categorize()`** - Classifies text into Islamic studies categories 
+- **`categorize()`** - Classifies text into given categories (You have to create a category tree)
 - **`extract_keywords()`** - Extracts keywords from text
 - **`extract_entities()`** - Named Entity Recognition (NER) system
 - **`is_question()`** - Binary detection of whether input is a question
@@ -64,7 +64,7 @@ Each tool is designed to work with structured outputs (JSON / Pydantic).
 
 ---
 
-## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature` and `validator` parameters
+## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature`, `validator` and `priority` parameters
 
 TextTools provides several optional flags to customize LLM behavior:
 
@@ -72,6 +72,7 @@ TextTools provides several optional flags to customize LLM behavior:
 **Note:** This doubles token usage per call because it triggers an additional LLM request.
 
 - **`logprobs (bool)`** → Returns token-level probabilities for the generated output. You can also specify `top_logprobs=<N>` to get the top N alternative tokens and their probabilities.  
+**Note:** This feature works if it's supported by the model.
 
 - **`output_lang (str)`** → Forces the model to respond in a specific language. The model will ignore other instructions about language and respond strictly in the requested language.
 
@@ -79,9 +80,10 @@ TextTools provides several optional flags to customize LLM behavior:
 
 - **`temperature (float)`** → Determines how creative the model should respond. Takes a float number from `0.0` to `2.0`.
 
-- **`validator (Callable)`** → Forces TheTool to validate the output result based on your custom validator. Validator should return bool (True if there were no problem, False if the validation failed.) If validator failed, TheTool will retry to get another output by modifying `temperature`. You can specify `max_validation_retries=<N>` to change the number of retries.
+- **`validator (Callable)`** → Forces TheTool to validate the output result based on your custom validator. Validator should return a bool (True if there were no problem, False if the validation fails.) If the validator fails, TheTool will retry to get another output by modifying `temperature`. You can specify `max_validation_retries=<N>` to change the number of retries.
 
-All these parameters can be used individually or together to tailor the behavior of any tool in **TextTools**.
+- **`priority (int)`** → Task execution priority level. Higher values = higher priority. Affects processing order in queues.
+**Note:** This feature works if it's supported by the model and vLLM.
 
 **Note:** There might be some tools that don't support some of the parameters above.
 
@@ -95,7 +97,7 @@ Every tool of `TextTools` returns a `ToolOutput` object which is a BaseModel wit
 - **`logprobs (list)`** → Token-level probabilities for the generated output 
 - **`errors (list[str])`** → Any error that have occured during calling LLM
 
-**None:** You can use `repr(ToolOutput)` to see details of an output.
+**Note:** You can use `repr(ToolOutput)` to see details of an output.
 
 ---
 

{hamtaa_texttools-1.1.13.dist-info → hamtaa_texttools-1.1.14.dist-info}/RECORD

@@ -1,13 +1,14 @@
-hamtaa_texttools-1.1.13.dist-info/licenses/LICENSE,sha256=Hb2YOBKy2MJQLnyLrX37B4ZVuac8eaIcE71SvVIMOLg,1082
-texttools/__init__.py,sha256=EZPPNPafVGvBaxjG9anP0piqH3gAC0DdjdAckQeAgNU,251
-texttools/batch/batch_config.py,sha256=FCDXy9TfH7xjd1PHvn_CtdwEQSq-YO5sktiaMZEId58,740
-texttools/batch/batch_runner.py,sha256=zzzVIXedmaq-8fqsFtGRR64F7CtYRLlhQeBu8uMwJQg,9385
+hamtaa_texttools-1.1.14.dist-info/licenses/LICENSE,sha256=Hb2YOBKy2MJQLnyLrX37B4ZVuac8eaIcE71SvVIMOLg,1082
+texttools/__init__.py,sha256=dc81lXGWP29k7oVvq2BMoMotz6lgiwX4PO2jHHBe2S8,317
+texttools/batch/batch_config.py,sha256=m1UgILVKjNdWE6laNbfbG4vgi4o2fEegGZbeoam6pnY,749
+texttools/batch/batch_runner.py,sha256=9e4SPLlvLHHs3U7bHkuuMVw8TFNwsGUzRjkAMKN4_ik,9378
 texttools/batch/internals/batch_manager.py,sha256=UoBe76vmFG72qrSaGKDZf4HzkykFBkkkbL9TLfV8TuQ,8730
 texttools/batch/internals/utils.py,sha256=F1_7YlVFKhjUROAFX4m0SaP8KiZVZyHRMIIB87VUGQc,373
 texttools/prompts/README.md,sha256=-5YO93CN93QLifqZpUeUnCOCBbDiOTV-cFQeJ7Gg0I4,1377
-texttools/prompts/categorizer.yaml,sha256=GMqIIzQFhgnlpkgU1qi3FAD3mD4A2jiWD5TilQ2XnnE,1204
+texttools/prompts/categorize.yaml,sha256=F7VezB25B_sT5yoC25ezODBddkuDD5lUHKetSpx9FKI,2743
+texttools/prompts/detect_entity.yaml,sha256=1rhMkJOjxSQcT4j_c5SRcIm77AUdeG-rUmeidb6VOFc,981
 texttools/prompts/extract_entities.yaml,sha256=KiKjeDpHaeh3JVtZ6q1pa3k4DYucUIU9WnEcRTCA-SE,651
-texttools/prompts/extract_keywords.yaml,sha256=0O7ypL_OsEOxtvlQ2CZjnsv9637DJwAKprZsf9Vo2_s,769
+texttools/prompts/extract_keywords.yaml,sha256=Vj4Tt3vT6LtpOo_iBZPo9oWI50oVdPGXe5i8yDR8ex4,3177
 texttools/prompts/is_question.yaml,sha256=d0-vKRbXWkxvO64ikvxRjEmpAXGpCYIPGhgexvPPjws,471
 texttools/prompts/merge_questions.yaml,sha256=0J85GvTirZB4ELwH3sk8ub_WcqqpYf6PrMKr3djlZeo,1792
 texttools/prompts/rewrite.yaml,sha256=LO7He_IA3MZKz8a-LxH9DHJpOjpYwaYN1pbjp1Y0tFo,5392
@@ -16,15 +17,15 @@ texttools/prompts/subject_to_question.yaml,sha256=C7x7rNNm6U_ZG9HOn6zuzYOtvJUZ2s
 texttools/prompts/summarize.yaml,sha256=o6rxGPfWtZd61Duvm8NVvCJqfq73b-wAuMSKR6UYUqY,459
 texttools/prompts/text_to_question.yaml,sha256=UheKYpDn6iyKI8NxunHZtFpNyfCLZZe5cvkuXpurUJY,783
 texttools/prompts/translate.yaml,sha256=mGT2uBCei6uucWqVbs4silk-UV060v3G0jnt0P6sr50,634
-texttools/tools/async_tools.py,sha256=60VAAZyVRxI2rKVFFiCnbY--F4kNtVxYQticE0RyhOs,24677
-texttools/tools/sync_tools.py,sha256=F5TN3KQ_vlF7AC9J0vm2NzjIZC19Ox11tpc9K1SMRwQ,24448
-texttools/tools/internals/async_operator.py,sha256=9OzF5FFXYrXX1C6ZDbad1zw9A6BZsDQ65jQVrpqTlPw,6961
+texttools/tools/async_tools.py,sha256=2bWwHd5eaUZwEt4u0rsl6K7QTpE0r83sd8nGaMqStyA,31994
+texttools/tools/sync_tools.py,sha256=DduJMkHtP0pLiEVbGtfuVq__kVhyMy8Hwi528OusyGc,31747
+texttools/tools/internals/async_operator.py,sha256=kVYClefPlcIhaMwcfh2rNX0GROaStjXdVe6bh_XTGKM,7149
 texttools/tools/internals/formatters.py,sha256=tACNLP6PeoqaRpNudVxBaHA25zyWqWYPZQuYysIu88g,941
+texttools/tools/internals/models.py,sha256=VH9l4d_ER8skBCTj5w9ISdRnsUXb3H2OVehz9r7VgdM,5630
 texttools/tools/internals/operator_utils.py,sha256=w1k0RJ_W_CRbVc_J2w337VuL-opHpHiCxfhEOwtyuOo,1856
-texttools/tools/internals/output_models.py,sha256=ekpbyocmXj_dee7ieOT1zOkMo9cPHT7xcUFCZoUaXA0,1886
 texttools/tools/internals/prompt_loader.py,sha256=4g6-U8kqrGN7VpNaRcrBcnF-h03PXjUDBP0lL0_4EZY,1953
-texttools/tools/internals/sync_operator.py,sha256=zbFLbFvaT9hAdIgpbDv17ljuqqu6ZeIOwCquM4gHTI8,6867
-hamtaa_texttools-1.1.13.dist-info/METADATA,sha256=4qkKZKb9DEb1vx2FMD-xRBDj_LCnDeytl-ea3zMVKdc,9179
-hamtaa_texttools-1.1.13.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-hamtaa_texttools-1.1.13.dist-info/top_level.txt,sha256=5Mh0jIxxZ5rOXHGJ6Mp-JPKviywwN0MYuH0xk5bEWqE,10
-hamtaa_texttools-1.1.13.dist-info/RECORD,,
+texttools/tools/internals/sync_operator.py,sha256=5kZWO8CGgoXx1FGFOWU1wnj2RWW8_KNEMGnQCkmbYwA,7057
+hamtaa_texttools-1.1.14.dist-info/METADATA,sha256=Smh1J0R1JjMkeNOwc-bIEsAVDexdERwRM3T6ZpBcDkk,9370
+hamtaa_texttools-1.1.14.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+hamtaa_texttools-1.1.14.dist-info/top_level.txt,sha256=5Mh0jIxxZ5rOXHGJ6Mp-JPKviywwN0MYuH0xk5bEWqE,10
+hamtaa_texttools-1.1.14.dist-info/RECORD,,

texttools/__init__.py

@@ -2,5 +2,6 @@ from .batch.batch_runner import BatchJobRunner
 from .batch.batch_config import BatchConfig
 from .tools.sync_tools import TheTool
 from .tools.async_tools import AsyncTheTool
+from .tools.internals.models import CategoryTree
 
-__all__ = ["TheTool", "AsyncTheTool", "BatchJobRunner", "BatchConfig"]
+__all__ = ["TheTool", "AsyncTheTool", "BatchJobRunner", "BatchConfig", "CategoryTree"]

texttools/batch/batch_config.py

@@ -1,5 +1,5 @@
 from dataclasses import dataclass
-from typing import Callable
+from collections.abc import Callable
 
 from texttools.batch.internals.utils import import_data, export_data
 

texttools/batch/batch_runner.py

@@ -11,7 +11,7 @@ from pydantic import BaseModel
 
 from texttools.batch.internals.batch_manager import BatchManager
 from texttools.batch.batch_config import BatchConfig
-from texttools.tools.internals.output_models import StrOutput
+from texttools.tools.internals.models import StrOutput
 
 # Base Model type for output models
 T = TypeVar("T", bound=BaseModel)

texttools/prompts/categorize.yaml

@@ -0,0 +1,77 @@
+main_template:
+
+  category_list: |
+    You are an expert classification agent.  
+    You receive a list of categories.
+
+    Your task:
+    - Read all provided categories carefully.
+    - Consider the user query, intent, and task explanation.
+    - Select exactly one category name from the list that best matches the user’s intent.
+    - Return only the category name, nothing else.
+
+    Rules:
+    - Never invent categories that are not in the list.
+    - If multiple categories seem possible, choose the closest match based on the description and user intent.
+    - If descriptions are missing or empty, rely on the category name.
+    - If the correct answer cannot be determined with certainty, choose the most likely one.
+
+    Output format:
+    {{
+    "reason": "Explanation of why the input belongs to the category"
+    "result": "<category_name_only>"
+    }}
+
+    Available categories with their descriptions:
+    {category_list}
+
+    The text that has to be categorized:
+    {input}
+
+  category_tree: |
+    You are an expert classification agent.  
+    You receive a list of categories at the current level of a hierarchical category tree.  
+
+    Your task:
+    - Read all provided categories carefully.
+    - Consider the user query, intent, and task explanation.
+    - Select exactly one category name from the list that best matches the user’s intent.
+    - Return only the category name, nothing else.
+
+    Rules:
+    - Never invent categories that are not in the list.
+    - If multiple categories seem possible, choose the closest match based on the description and user intent.
+    - If descriptions are missing or empty, rely on the category name.
+    - If the correct answer cannot be determined with certainty, choose the most likely one.
+
+    Output format:
+    {{
+    "reason": "Explanation of why the input belongs to the category"
+    "result": "<category_name_only>"
+    }}
+
+    Available categories with their descriptions at this level:
+    {category_list}
+
+    Do not include category descriptions at all. Only write the raw category.
+
+    The text that has to be categorized:
+    {input}
+
+analyze_template:
+
+  category_list: |
+    We want to categorize the given text.
+    To improve categorization, we need an analysis of the text.
+    Analyze the given text and write its main idea and a short analysis of that.
+    Analysis should be very short.
+    Text:
+    {input}
+
+  category_tree: |
+    We want to categorize the given text.
+    To improve categorization, we need an analysis of the text.
+    Analyze the given text and write its main idea and a short analysis of that.
+    Analysis should be very short.
+    Text:
+    {input}

texttools/prompts/detect_entity.yaml

@@ -0,0 +1,22 @@
+main_template: |
+  You are an expert Named Entity Recognition (NER) system. Extract entities from the text.
+  The output must strictly follow the provided Pydantic schema.
+  
+  Mapping Rule:
+  - Person: شخص
+  - Location: مکان
+  - Time: زمان
+  - Living Beings: موجود زنده
+  - Organization: سازمان
+  - Concept: مفهوم
+  
+  CRITICAL:
+  1. The final output structure must be a complete JSON object matching the Pydantic schema (List[Entity]).
+  2. Both the extracted text and the type must be in Persian, using the exact mapping provided above.
+  
+  Here is the text: {input}
+
+analyze_template: |
+  Analyze the following text to identify all potential named entities and their categories (Person, Location, Time, Living Beings, Organization, Concept). 
+  Provide a brief summary of the entities identified that will help the main process to extract them accurately and apply the correct Persian type label.
+  Here is the text: {input}

texttools/prompts/extract_keywords.yaml

@@ -1,18 +1,68 @@
-main_template: |
-  You are an expert keyword extractor.
-  Extract the most relevant keywords from the given text.
-  Guidelines:
-  - Keywords must represent the main concepts of the text.  
-  - If two words have overlapping meanings, choose only one.  
-  - Do not include generic or unrelated words.  
-  - Keywords must be single, self-contained words (no phrases).
-  - Output between 3 and 7 keywords based on the input length.
-  - Respond only in JSON format:
-  {{"result": ["keyword1", "keyword2", etc.]}}
-  Here is the text:
-  {input}
-
-analyze_template: |
-  Analyze the following text to identify its main topics, concepts, and important terms.
-  Provide a concise summary of your findings that will help in extracting relevant keywords.
-  {input}
+main_template:
+
+  auto: |
+    You are an expert keyword extractor.
+    Extract the most relevant keywords from the given text.
+    Guidelines:
+    - Keywords must represent the main concepts of the text.  
+    - If two words have overlapping meanings, choose only one.  
+    - Do not include generic or unrelated words.  
+    - Keywords must be single, self-contained words (no phrases).
+    - Output between 3 and 7 keywords based on the input length.
+    - Respond only in JSON format:
+    {{"result": ["keyword1", "keyword2", etc.]}}
+    Here is the text:
+    {input}
+
+  threshold: |
+    You are an expert keyword extractor specialized in fine-grained concept identification.
+    Extract the most specific, content-bearing keywords from the text.
+
+    Requirements:
+    - Choose fine-grained conceptual terms, not general domain labels.
+    - Avoid words that only describe the broad topic (e.g., Islam, religion, philosophy, history).
+    - Prefer specific names, concepts, doctrines, events, arguments, or terminology.
+    - Do not select words only because they appear frequently. A keyword must represent a central conceptual idea, not a repeated surface term.
+    - If multiple words express overlapping meaning, select the more specific one.
+    - Keywords must be single words (no multi-word expressions).
+    - Extract N keywords depending on input length:
+      - Short texts (a few sentences): 3 keywords
+      - Medium texts (1–4 paragraphs): 4–5 keywords
+      - Long texts (more than 4 paragraphs): 6–7 keywords
+    - Respond only in JSON format:
+    {{"result": ["keyword1", "keyword2", etc.]}}
+    Here is the text:
+    {input}
+
+  count: |
+    You are an expert keyword extractor with precise output requirements.
+    Extract exactly {number_of_keywords} keywords from the given text.
+
+    Requirements:
+    - Extract exactly {number_of_keywords} keywords, no more, no less.
+    - Select the {number_of_keywords} most relevant and specific keywords that represent core concepts.
+    - Prefer specific terms, names, and concepts over general topic labels.
+    - If the text doesn't contain enough distinct keywords, include the most relevant ones even if some are less specific.
+    - Keywords must be single words (no multi-word expressions).
+    - Order keywords by relevance (most relevant first).
+    - Respond only in JSON format:
+    {{"result": ["keyword1", "keyword2", "keyword3", ...]}}
+    
+    Here is the text:
+    {input}
+
+analyze_template:
+  auto: |
+    Analyze the following text to identify its main topics, concepts, and important terms.
+    Provide a concise summary of your findings that will help in extracting relevant keywords.
+    {input}
+
+  threshold: |
+    Analyze the following text to identify its main topics, concepts, and important terms.
+    Provide a concise summary of your findings that will help in extracting relevant keywords.
+    {input}
+
+  count: |
+    Analyze the following text to identify its main topics, concepts, and important terms.
+    Provide a concise summary of your findings that will help in extracting relevant keywords.
+    {input}