hamtaa-texttools 1.1.20__tar.gz → 1.1.22__tar.gz

{hamtaa_texttools-1.1.20/hamtaa_texttools.egg-info → hamtaa_texttools-1.1.22}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: hamtaa-texttools
-Version: 1.1.20
+Version: 1.1.22
 Summary: A high-level NLP toolkit built on top of modern LLMs.
 Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>, Zareshahi <a.zareshahi1377@gmail.com>
 License: MIT License
@@ -37,61 +37,53 @@ Dynamic: license-file
 
 ## 📌 Overview
 
-**TextTools** is a high-level **NLP toolkit** built on top of modern **LLMs**.  
+**TextTools** is a high-level **NLP toolkit** built on top of **LLMs**.  
 
 It provides both **sync (`TheTool`)** and **async (`AsyncTheTool`)** APIs for maximum flexibility.
 
 It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extraction, and more** - designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
+**Note:** Most features of `texttools` are reliable when you use `google/gemma-3n-e4b-it` model.
+
 ---
 
 ## ✨ Features
 
 TextTools provides a rich collection of high-level NLP utilities,
-Each tool is designed to work with structured outputs (JSON / Pydantic).
+Each tool is designed to work with structured outputs.
 
-- **`categorize()`** - Classifies text into given categories (You have to create a category tree)
-- **`extract_keywords()`** - Extracts keywords from text
+- **`categorize()`** - Classifies text into given categories
+- **`extract_keywords()`** - Extracts keywords from the text
 - **`extract_entities()`** - Named Entity Recognition (NER) system
-- **`is_question()`** - Binary detection of whether input is a question
+- **`is_question()`** - Binary question detection
 - **`text_to_question()`** - Generates questions from text
-- **`merge_questions()`** - Merges multiple questions with different modes
-- **`rewrite()`** - Rewrites text with different wording/meaning
+- **`merge_questions()`** - Merges multiple questions into one
+- **`rewrite()`** - Rewrites text in a diffrent way
 - **`subject_to_question()`** - Generates questions about a specific subject
 - **`summarize()`** - Text summarization
-- **`translate()`** - Text translation between languages
+- **`translate()`** - Text translation
 - **`propositionize()`** - Convert text to atomic independence meaningful sentences 
-- **`check_fact()`** - Check a statement is relevant to source text or not 
+- **`check_fact()`** - Check whether a statement is relevant to the source text
 - **`run_custom()`** - Allows users to define a custom tool with an arbitrary BaseModel
 
 ---
 
+## 🚀 Installation
+
+Install the latest release via PyPI:
+
+```bash
+pip install -U hamtaa-texttools
+```
+
+---
+
 ## 📊 Tool Quality Tiers
 
-| Status | Meaning | Use in Production? |
-|--------|---------|-------------------|
-| **✅ Production** | Evaluated, tested, stable. | **Yes** - ready for reliable use. |
-| **🧪 Experimental** | Added to the package but **not fully evaluated**. Functional, but quality may vary. | **Use with caution** - outputs not yet validated. |
-
-### Current Status
-**Production Tools:**
-- `categorize()` (list mode)
-- `extract_keywords()`
-- `extract_entities()`
-- `is_question()`
-- `text_to_question()`
-- `merge_questions()`
-- `rewrite()`
-- `subject_to_question()`
-- `summarize()`
-- `run_custom()` (fine in most cases)
-
-**Experimental Tools:**
-- `categorize()` (tree mode)
-- `translate()`
-- `propositionize()`
-- `check_fact()`
-- `run_custom()` (not evaluated in all scenarios)
+| Status | Meaning | Tools | Use in Production? |
+|--------|---------|----------|-------------------|
+| **✅ Production** | Evaluated, tested, stable. | `categorize()` (list mode), `extract_keywords()`, `extract_entities()`, `is_question()`, `text_to_question()`, `merge_questions()`, `rewrite()`, `subject_to_question()`, `summarize()`, `run_custom()` | **Yes** - ready for reliable use. |
+| **🧪 Experimental** | Added to the package but **not fully evaluated**. Functional, but quality may vary. | `categorize()` (tree mode), `translate()`, `propositionize()`, `check_fact()` | **Use with caution** - outputs not yet validated. |
 
 ---
 
@@ -100,48 +92,37 @@ Each tool is designed to work with structured outputs (JSON / Pydantic).
 TextTools provides several optional flags to customize LLM behavior:
 
 - **`with_analysis: bool`** → Adds a reasoning step before generating the final output.
-**Note:** This doubles token usage per call because it triggers an additional LLM request.
+**Note:** This doubles token usage per call.
 
 - **`logprobs: bool`** → Returns token-level probabilities for the generated output. You can also specify `top_logprobs=<N>` to get the top N alternative tokens and their probabilities.  
 **Note:** This feature works if it's supported by the model.
 
-- **`output_lang: str`** → Forces the model to respond in a specific language. The model will ignore other instructions about language and respond strictly in the requested language.
+- **`output_lang: str`** → Forces the model to respond in a specific language.
 
-- **`user_prompt: str`** → Allows you to inject a custom instruction or prompt into the model alongside the main template. This gives you fine-grained control over how the model interprets or modifies the input text.
+- **`user_prompt: str`** → Allows you to inject a custom instruction or into the model alongside the main template. This gives you fine-grained control over how the model interprets or modifies the input text.
 
 - **`temperature: float`** → Determines how creative the model should respond. Takes a float number from `0.0` to `2.0`.
 
-- **`validator: Callable (Experimental)`** → Forces TheTool to validate the output result based on your custom validator. Validator should return a bool (True if there were no problem, False if the validation fails.) If the validator fails, TheTool will retry to get another output by modifying `temperature`. You can specify `max_validation_retries=<N>` to change the number of retries.
+- **`validator: Callable (Experimental)`** → Forces TheTool to validate the output result based on your custom validator. Validator should return a boolean. If the validator fails, TheTool will retry to get another output by modifying `temperature`. You can also specify `max_validation_retries=<N>`.
 
-- **`priority: int (Experimental)`** → Task execution priority level. Higher values = higher priority. Affects processing order in queues.
+- **`priority: int (Experimental)`** → Task execution priority level. Affects processing order in queues.
 **Note:** This feature works if it's supported by the model and vLLM.
 
-**Note:** There might be some tools that don't support some of the parameters above.
-
 ---
 
 ## 🧩 ToolOutput
 
 Every tool of `TextTools` returns a `ToolOutput` object which is a BaseModel with attributes:
-- **`result: Any`** → The output of LLM
-- **`analysis: str`** → The reasoning step before generating the final output
-- **`logprobs: list`** → Token-level probabilities for the generated output 
-- **`process: str`** → The tool name which processed the input
-- **`processed_at: datetime`** → The process time
-- **`execution_time: float`** → The execution time (seconds)
-- **`errors: list[str]`** → Any error that have occured during calling LLM
+- **`result: Any`**
+- **`analysis: str`**
+- **`logprobs: list`**
+- **`errors: list[str]`**
+- **`ToolOutputMetadata`** →
+    - **`tool_name: str`**
+    - **`processed_at: datetime`**
+    - **`execution_time: float`**
 
-**Note:** You can use `repr(ToolOutput)` to see details of your ToolOutput.
-
----
-
-## 🚀 Installation
-
-Install the latest release via PyPI:
-
-```bash
-pip install -U hamtaa-texttools
-```
+**Note:** You can use `repr(ToolOutput)` to print your output with all the details.
 
 ---
 
@@ -159,26 +140,13 @@ pip install -U hamtaa-texttools
 from openai import OpenAI
 from texttools import TheTool
 
-# Create your OpenAI client
 client = OpenAI(base_url = "your_url", API_KEY = "your_api_key")
+model = "model_name"
 
-# Specify the model
-model = "gpt-4o-mini"
-
-# Create an instance of TheTool
 the_tool = TheTool(client=client, model=model)
 
-# Example: Question Detection
-detection = the_tool.is_question("Is this project open source?", logprobs=True, top_logprobs=2)
-print(detection.result)
-print(detection.logprobs)
-# Output: True + logprobs
-
-# Example: Translation
-translation = the_tool.translate("سلام، حالت چطوره؟" target_language="English", with_analysis=True)
-print(translation.result)
-print(translation.analysis)
-# Output: "Hi! How are you?"  + analysis
+detection = the_tool.is_question("Is this project open source?")
+print(repr(detection))
 ```
 
 ---
@@ -191,22 +159,17 @@ from openai import AsyncOpenAI
 from texttools import AsyncTheTool
 
 async def main():
-    # Create your AsyncOpenAI client
     async_client = AsyncOpenAI(base_url="your_url", api_key="your_api_key")
+    model = "model_name"
 
-    # Specify the model
-    model = "gpt-4o-mini"
-
-    # Create an instance of AsyncTheTool
     async_the_tool = AsyncTheTool(client=async_client, model=model)
     
-    # Example: Async Translation and Keyword Extraction
     translation_task = async_the_tool.translate("سلام، حالت چطوره؟", target_language="English")
     keywords_task = async_the_tool.extract_keywords("Tomorrow, we will be dead by the car crash")
 
     (translation, keywords) = await asyncio.gather(translation_task, keywords_task)
-    print(translation.result)
-    print(keywords.result)
+    print(repr(translation))
+    print(repr(keywords))
 
 asyncio.run(main())
 ```
@@ -224,37 +187,16 @@ Use **TextTools** when you need to:
 
 ---
 
-## 🔍 Logging
-
-TextTools uses Python's standard `logging` module. The library's default logger level is `WARNING`, so if you want to modify it, follow instructions:
-
-
-```python
-import logging
-
-# Default: warnings and errors only
-logging.basicConfig(level=logging.WARNING)
-
-# Debug everything (verbose)
-logging.basicConfig(level=logging.DEBUG)
-
-# Complete silence
-logging.basicConfig(level=logging.CRITICAL)
-```
-
----
-
 ## 📚 Batch Processing
 
 Process large datasets efficiently using OpenAI's batch API.
 
-## ⚡ Quick Start (Batch)
+## ⚡ Quick Start (Batch Runner)
 
 ```python
 from pydantic import BaseModel
-from texttools import BatchJobRunner, BatchConfig
+from texttools import BatchRunner, BatchConfig
 
-# Configure your batch job
 config = BatchConfig(
     system_prompt="Extract entities from the text",
     job_name="entity_extraction",
@@ -263,12 +205,10 @@ config = BatchConfig(
     model="gpt-4o-mini"
 )
 
-# Define your output schema
 class Output(BaseModel):
     entities: list[str]
 
-# Run the batch job
-runner = BatchJobRunner(config, output_model=Output)
+runner = BatchRunner(config, output_model=Output)
 runner.run()
 ```
 

{hamtaa_texttools-1.1.20 → hamtaa_texttools-1.1.22}/README.md

@@ -2,61 +2,53 @@
 
 ## 📌 Overview
 
-**TextTools** is a high-level **NLP toolkit** built on top of modern **LLMs**.  
+**TextTools** is a high-level **NLP toolkit** built on top of **LLMs**.  
 
 It provides both **sync (`TheTool`)** and **async (`AsyncTheTool`)** APIs for maximum flexibility.
 
 It provides ready-to-use utilities for **translation, question detection, keyword extraction, categorization, NER extraction, and more** - designed to help you integrate AI-powered text processing into your applications with minimal effort.
 
+**Note:** Most features of `texttools` are reliable when you use `google/gemma-3n-e4b-it` model.
+
 ---
 
 ## ✨ Features
 
 TextTools provides a rich collection of high-level NLP utilities,
-Each tool is designed to work with structured outputs (JSON / Pydantic).
+Each tool is designed to work with structured outputs.
 
-- **`categorize()`** - Classifies text into given categories (You have to create a category tree)
-- **`extract_keywords()`** - Extracts keywords from text
+- **`categorize()`** - Classifies text into given categories
+- **`extract_keywords()`** - Extracts keywords from the text
 - **`extract_entities()`** - Named Entity Recognition (NER) system
-- **`is_question()`** - Binary detection of whether input is a question
+- **`is_question()`** - Binary question detection
 - **`text_to_question()`** - Generates questions from text
-- **`merge_questions()`** - Merges multiple questions with different modes
-- **`rewrite()`** - Rewrites text with different wording/meaning
+- **`merge_questions()`** - Merges multiple questions into one
+- **`rewrite()`** - Rewrites text in a diffrent way
 - **`subject_to_question()`** - Generates questions about a specific subject
 - **`summarize()`** - Text summarization
-- **`translate()`** - Text translation between languages
+- **`translate()`** - Text translation
 - **`propositionize()`** - Convert text to atomic independence meaningful sentences 
-- **`check_fact()`** - Check a statement is relevant to source text or not 
+- **`check_fact()`** - Check whether a statement is relevant to the source text
 - **`run_custom()`** - Allows users to define a custom tool with an arbitrary BaseModel
 
 ---
 
+## 🚀 Installation
+
+Install the latest release via PyPI:
+
+```bash
+pip install -U hamtaa-texttools
+```
+
+---
+
 ## 📊 Tool Quality Tiers
 
-| Status | Meaning | Use in Production? |
-|--------|---------|-------------------|
-| **✅ Production** | Evaluated, tested, stable. | **Yes** - ready for reliable use. |
-| **🧪 Experimental** | Added to the package but **not fully evaluated**. Functional, but quality may vary. | **Use with caution** - outputs not yet validated. |
-
-### Current Status
-**Production Tools:**
-- `categorize()` (list mode)
-- `extract_keywords()`
-- `extract_entities()`
-- `is_question()`
-- `text_to_question()`
-- `merge_questions()`
-- `rewrite()`
-- `subject_to_question()`
-- `summarize()`
-- `run_custom()` (fine in most cases)
-
-**Experimental Tools:**
-- `categorize()` (tree mode)
-- `translate()`
-- `propositionize()`
-- `check_fact()`
-- `run_custom()` (not evaluated in all scenarios)
+| Status | Meaning | Tools | Use in Production? |
+|--------|---------|----------|-------------------|
+| **✅ Production** | Evaluated, tested, stable. | `categorize()` (list mode), `extract_keywords()`, `extract_entities()`, `is_question()`, `text_to_question()`, `merge_questions()`, `rewrite()`, `subject_to_question()`, `summarize()`, `run_custom()` | **Yes** - ready for reliable use. |
+| **🧪 Experimental** | Added to the package but **not fully evaluated**. Functional, but quality may vary. | `categorize()` (tree mode), `translate()`, `propositionize()`, `check_fact()` | **Use with caution** - outputs not yet validated. |
 
 ---
 
@@ -65,48 +57,37 @@ Each tool is designed to work with structured outputs (JSON / Pydantic).
 TextTools provides several optional flags to customize LLM behavior:
 
 - **`with_analysis: bool`** → Adds a reasoning step before generating the final output.
-**Note:** This doubles token usage per call because it triggers an additional LLM request.
+**Note:** This doubles token usage per call.
 
 - **`logprobs: bool`** → Returns token-level probabilities for the generated output. You can also specify `top_logprobs=<N>` to get the top N alternative tokens and their probabilities.  
 **Note:** This feature works if it's supported by the model.
 
-- **`output_lang: str`** → Forces the model to respond in a specific language. The model will ignore other instructions about language and respond strictly in the requested language.
+- **`output_lang: str`** → Forces the model to respond in a specific language.
 
-- **`user_prompt: str`** → Allows you to inject a custom instruction or prompt into the model alongside the main template. This gives you fine-grained control over how the model interprets or modifies the input text.
+- **`user_prompt: str`** → Allows you to inject a custom instruction or into the model alongside the main template. This gives you fine-grained control over how the model interprets or modifies the input text.
 
 - **`temperature: float`** → Determines how creative the model should respond. Takes a float number from `0.0` to `2.0`.
 
-- **`validator: Callable (Experimental)`** → Forces TheTool to validate the output result based on your custom validator. Validator should return a bool (True if there were no problem, False if the validation fails.) If the validator fails, TheTool will retry to get another output by modifying `temperature`. You can specify `max_validation_retries=<N>` to change the number of retries.
+- **`validator: Callable (Experimental)`** → Forces TheTool to validate the output result based on your custom validator. Validator should return a boolean. If the validator fails, TheTool will retry to get another output by modifying `temperature`. You can also specify `max_validation_retries=<N>`.
 
-- **`priority: int (Experimental)`** → Task execution priority level. Higher values = higher priority. Affects processing order in queues.
+- **`priority: int (Experimental)`** → Task execution priority level. Affects processing order in queues.
 **Note:** This feature works if it's supported by the model and vLLM.
 
-**Note:** There might be some tools that don't support some of the parameters above.
-
 ---
 
 ## 🧩 ToolOutput
 
 Every tool of `TextTools` returns a `ToolOutput` object which is a BaseModel with attributes:
-- **`result: Any`** → The output of LLM
-- **`analysis: str`** → The reasoning step before generating the final output
-- **`logprobs: list`** → Token-level probabilities for the generated output 
-- **`process: str`** → The tool name which processed the input
-- **`processed_at: datetime`** → The process time
-- **`execution_time: float`** → The execution time (seconds)
-- **`errors: list[str]`** → Any error that have occured during calling LLM
+- **`result: Any`**
+- **`analysis: str`**
+- **`logprobs: list`**
+- **`errors: list[str]`**
+- **`ToolOutputMetadata`** →
+    - **`tool_name: str`**
+    - **`processed_at: datetime`**
+    - **`execution_time: float`**
 
-**Note:** You can use `repr(ToolOutput)` to see details of your ToolOutput.
-
----
-
-## 🚀 Installation
-
-Install the latest release via PyPI:
-
-```bash
-pip install -U hamtaa-texttools
-```
+**Note:** You can use `repr(ToolOutput)` to print your output with all the details.
 
 ---
 
@@ -124,26 +105,13 @@ pip install -U hamtaa-texttools
 from openai import OpenAI
 from texttools import TheTool
 
-# Create your OpenAI client
 client = OpenAI(base_url = "your_url", API_KEY = "your_api_key")
+model = "model_name"
 
-# Specify the model
-model = "gpt-4o-mini"
-
-# Create an instance of TheTool
 the_tool = TheTool(client=client, model=model)
 
-# Example: Question Detection
-detection = the_tool.is_question("Is this project open source?", logprobs=True, top_logprobs=2)
-print(detection.result)
-print(detection.logprobs)
-# Output: True + logprobs
-
-# Example: Translation
-translation = the_tool.translate("سلام، حالت چطوره؟" target_language="English", with_analysis=True)
-print(translation.result)
-print(translation.analysis)
-# Output: "Hi! How are you?"  + analysis
+detection = the_tool.is_question("Is this project open source?")
+print(repr(detection))
 ```
 
 ---
@@ -156,22 +124,17 @@ from openai import AsyncOpenAI
 from texttools import AsyncTheTool
 
 async def main():
-    # Create your AsyncOpenAI client
     async_client = AsyncOpenAI(base_url="your_url", api_key="your_api_key")
+    model = "model_name"
 
-    # Specify the model
-    model = "gpt-4o-mini"
-
-    # Create an instance of AsyncTheTool
     async_the_tool = AsyncTheTool(client=async_client, model=model)
     
-    # Example: Async Translation and Keyword Extraction
     translation_task = async_the_tool.translate("سلام، حالت چطوره؟", target_language="English")
     keywords_task = async_the_tool.extract_keywords("Tomorrow, we will be dead by the car crash")
 
     (translation, keywords) = await asyncio.gather(translation_task, keywords_task)
-    print(translation.result)
-    print(keywords.result)
+    print(repr(translation))
+    print(repr(keywords))
 
 asyncio.run(main())
 ```
@@ -189,37 +152,16 @@ Use **TextTools** when you need to:
 
 ---
 
-## 🔍 Logging
-
-TextTools uses Python's standard `logging` module. The library's default logger level is `WARNING`, so if you want to modify it, follow instructions:
-
-
-```python
-import logging
-
-# Default: warnings and errors only
-logging.basicConfig(level=logging.WARNING)
-
-# Debug everything (verbose)
-logging.basicConfig(level=logging.DEBUG)
-
-# Complete silence
-logging.basicConfig(level=logging.CRITICAL)
-```
-
----
-
 ## 📚 Batch Processing
 
 Process large datasets efficiently using OpenAI's batch API.
 
-## ⚡ Quick Start (Batch)
+## ⚡ Quick Start (Batch Runner)
 
 ```python
 from pydantic import BaseModel
-from texttools import BatchJobRunner, BatchConfig
+from texttools import BatchRunner, BatchConfig
 
-# Configure your batch job
 config = BatchConfig(
     system_prompt="Extract entities from the text",
     job_name="entity_extraction",
@@ -228,12 +170,10 @@ config = BatchConfig(
     model="gpt-4o-mini"
 )
 
-# Define your output schema
 class Output(BaseModel):
     entities: list[str]
 
-# Run the batch job
-runner = BatchJobRunner(config, output_model=Output)
+runner = BatchRunner(config, output_model=Output)
 runner.run()
 ```