hamtaa-texttools 1.1.18__tar.gz → 1.1.19__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. {hamtaa_texttools-1.1.18/hamtaa_texttools.egg-info → hamtaa_texttools-1.1.19}/PKG-INFO +31 -1
  2. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/README.md +30 -0
  3. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19/hamtaa_texttools.egg-info}/PKG-INFO +31 -1
  4. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/hamtaa_texttools.egg-info/SOURCES.txt +1 -1
  5. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/pyproject.toml +1 -1
  6. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/tests/test_all_async_tools.py +4 -0
  7. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/tests/test_all_tools.py +7 -0
  8. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/internals/prompt_loader.py +34 -6
  9. hamtaa_texttools-1.1.19/texttools/prompts/check_fact.yaml +19 -0
  10. hamtaa_texttools-1.1.19/texttools/prompts/propositionize.yaml +22 -0
  11. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/run_custom.yaml +1 -1
  12. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/tools/async_tools.py +31 -13
  13. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/tools/sync_tools.py +31 -13
  14. hamtaa_texttools-1.1.18/texttools/prompts/detect_entity.yaml +0 -22
  15. hamtaa_texttools-1.1.18/texttools/prompts/propositionize.yaml +0 -15
  16. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/LICENSE +0 -0
  17. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/MANIFEST.in +0 -0
  18. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/hamtaa_texttools.egg-info/dependency_links.txt +0 -0
  19. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/hamtaa_texttools.egg-info/requires.txt +0 -0
  20. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/hamtaa_texttools.egg-info/top_level.txt +0 -0
  21. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/setup.cfg +0 -0
  22. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/tests/test_output_validation.py +0 -0
  23. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/__init__.py +0 -0
  24. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/batch/batch_config.py +0 -0
  25. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/batch/batch_runner.py +0 -0
  26. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/batch/internals/batch_manager.py +0 -0
  27. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/batch/internals/utils.py +0 -0
  28. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/internals/async_operator.py +0 -0
  29. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/internals/exceptions.py +0 -0
  30. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/internals/formatters.py +0 -0
  31. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/internals/models.py +0 -0
  32. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/internals/operator_utils.py +0 -0
  33. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/internals/sync_operator.py +0 -0
  34. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/README.md +0 -0
  35. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/categorize.yaml +0 -0
  36. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/extract_entities.yaml +0 -0
  37. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/extract_keywords.yaml +0 -0
  38. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/is_question.yaml +0 -0
  39. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/merge_questions.yaml +0 -0
  40. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/rewrite.yaml +0 -0
  41. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/subject_to_question.yaml +0 -0
  42. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/summarize.yaml +0 -0
  43. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/text_to_question.yaml +0 -0
  44. {hamtaa_texttools-1.1.18 → hamtaa_texttools-1.1.19}/texttools/prompts/translate.yaml +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: hamtaa-texttools
3
- Version: 1.1.18
3
+ Version: 1.1.19
4
4
  Summary: A high-level NLP toolkit built on top of modern LLMs.
5
5
  Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>, Zareshahi <a.zareshahi1377@gmail.com>
6
6
  License: MIT License
@@ -61,10 +61,40 @@ Each tool is designed to work with structured outputs (JSON / Pydantic).
61
61
  - **`summarize()`** - Text summarization
62
62
  - **`translate()`** - Text translation between languages
63
63
  - **`propositionize()`** - Convert text to atomic independence meaningful sentences
64
+ - **`check_fact()`** - Check a statement is relevant to source text or not
64
65
  - **`run_custom()`** - Allows users to define a custom tool with an arbitrary BaseModel
65
66
 
66
67
  ---
67
68
 
69
+ ## 📊 Tool Quality Tiers
70
+
71
+ | Status | Meaning | Use in Production? |
72
+ |--------|---------|-------------------|
73
+ | **✅ Production** | Evaluated, tested, stable. | **Yes** - ready for reliable use. |
74
+ | **🧪 Experimental** | Added to the package but **not fully evaluated**. Functional, but quality may vary. | **Use with caution** - outputs not yet validated. |
75
+
76
+ ### Current Status
77
+ **Production Tools:**
78
+ - `categorize()` (list mode)
79
+ - `extract_keywords()`
80
+ - `extract_entities()`
81
+ - `is_question()`
82
+ - `text_to_question()`
83
+ - `merge_questions()`
84
+ - `rewrite()`
85
+ - `subject_to_question()`
86
+ - `summarize()`
87
+ - `run_custom()` (fine in most cases)
88
+
89
+ **Experimental Tools:**
90
+ - `categorize()` (tree mode)
91
+ - `translate()`
92
+ - `propositionize()`
93
+ - `check_fact()`
94
+ - `run_custom()` (not evaluated in all scenarios)
95
+
96
+ ---
97
+
68
98
  ## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature`, `validator` and `priority` parameters
69
99
 
70
100
  TextTools provides several optional flags to customize LLM behavior:
@@ -26,10 +26,40 @@ Each tool is designed to work with structured outputs (JSON / Pydantic).
26
26
  - **`summarize()`** - Text summarization
27
27
  - **`translate()`** - Text translation between languages
28
28
  - **`propositionize()`** - Convert text to atomic independence meaningful sentences
29
+ - **`check_fact()`** - Check a statement is relevant to source text or not
29
30
  - **`run_custom()`** - Allows users to define a custom tool with an arbitrary BaseModel
30
31
 
31
32
  ---
32
33
 
34
+ ## 📊 Tool Quality Tiers
35
+
36
+ | Status | Meaning | Use in Production? |
37
+ |--------|---------|-------------------|
38
+ | **✅ Production** | Evaluated, tested, stable. | **Yes** - ready for reliable use. |
39
+ | **🧪 Experimental** | Added to the package but **not fully evaluated**. Functional, but quality may vary. | **Use with caution** - outputs not yet validated. |
40
+
41
+ ### Current Status
42
+ **Production Tools:**
43
+ - `categorize()` (list mode)
44
+ - `extract_keywords()`
45
+ - `extract_entities()`
46
+ - `is_question()`
47
+ - `text_to_question()`
48
+ - `merge_questions()`
49
+ - `rewrite()`
50
+ - `subject_to_question()`
51
+ - `summarize()`
52
+ - `run_custom()` (fine in most cases)
53
+
54
+ **Experimental Tools:**
55
+ - `categorize()` (tree mode)
56
+ - `translate()`
57
+ - `propositionize()`
58
+ - `check_fact()`
59
+ - `run_custom()` (not evaluated in all scenarios)
60
+
61
+ ---
62
+
33
63
  ## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature`, `validator` and `priority` parameters
34
64
 
35
65
  TextTools provides several optional flags to customize LLM behavior:
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: hamtaa-texttools
3
- Version: 1.1.18
3
+ Version: 1.1.19
4
4
  Summary: A high-level NLP toolkit built on top of modern LLMs.
5
5
  Author-email: Tohidi <the.mohammad.tohidi@gmail.com>, Montazer <montazerh82@gmail.com>, Givechi <mohamad.m.givechi@gmail.com>, MoosaviNejad <erfanmoosavi84@gmail.com>, Zareshahi <a.zareshahi1377@gmail.com>
6
6
  License: MIT License
@@ -61,10 +61,40 @@ Each tool is designed to work with structured outputs (JSON / Pydantic).
61
61
  - **`summarize()`** - Text summarization
62
62
  - **`translate()`** - Text translation between languages
63
63
  - **`propositionize()`** - Convert text to atomic independence meaningful sentences
64
+ - **`check_fact()`** - Check a statement is relevant to source text or not
64
65
  - **`run_custom()`** - Allows users to define a custom tool with an arbitrary BaseModel
65
66
 
66
67
  ---
67
68
 
69
+ ## 📊 Tool Quality Tiers
70
+
71
+ | Status | Meaning | Use in Production? |
72
+ |--------|---------|-------------------|
73
+ | **✅ Production** | Evaluated, tested, stable. | **Yes** - ready for reliable use. |
74
+ | **🧪 Experimental** | Added to the package but **not fully evaluated**. Functional, but quality may vary. | **Use with caution** - outputs not yet validated. |
75
+
76
+ ### Current Status
77
+ **Production Tools:**
78
+ - `categorize()` (list mode)
79
+ - `extract_keywords()`
80
+ - `extract_entities()`
81
+ - `is_question()`
82
+ - `text_to_question()`
83
+ - `merge_questions()`
84
+ - `rewrite()`
85
+ - `subject_to_question()`
86
+ - `summarize()`
87
+ - `run_custom()` (fine in most cases)
88
+
89
+ **Experimental Tools:**
90
+ - `categorize()` (tree mode)
91
+ - `translate()`
92
+ - `propositionize()`
93
+ - `check_fact()`
94
+ - `run_custom()` (not evaluated in all scenarios)
95
+
96
+ ---
97
+
68
98
  ## ⚙️ `with_analysis`, `logprobs`, `output_lang`, `user_prompt`, `temperature`, `validator` and `priority` parameters
69
99
 
70
100
  TextTools provides several optional flags to customize LLM behavior:
@@ -24,7 +24,7 @@ texttools/internals/prompt_loader.py
24
24
  texttools/internals/sync_operator.py
25
25
  texttools/prompts/README.md
26
26
  texttools/prompts/categorize.yaml
27
- texttools/prompts/detect_entity.yaml
27
+ texttools/prompts/check_fact.yaml
28
28
  texttools/prompts/extract_entities.yaml
29
29
  texttools/prompts/extract_keywords.yaml
30
30
  texttools/prompts/is_question.yaml
@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
4
4
 
5
5
  [project]
6
6
  name = "hamtaa-texttools"
7
- version = "1.1.18"
7
+ version = "1.1.19"
8
8
  authors = [
9
9
  { name = "Tohidi", email = "the.mohammad.tohidi@gmail.com" },
10
10
  { name = "Montazer", email = "montazerh82@gmail.com" },
@@ -47,6 +47,7 @@ async def main():
47
47
  "جنگ جهانی دوم در سال ۱۹۳۹ آغاز شد و آلمان به لهستان حمله کرد.",
48
48
  output_lang="Persian",
49
49
  )
50
+ check_fact_task = t.check_fact(text="امام نهم در ایران به خاک سپرده شد", source_text="حرم مطهر امام رضا علیه السلام در مشهد مقدس هست")
50
51
  (
51
52
  category,
52
53
  keywords,
@@ -59,6 +60,7 @@ async def main():
59
60
  summary,
60
61
  translation,
61
62
  propositionize,
63
+ check_fact,
62
64
  ) = await asyncio.gather(
63
65
  category_task,
64
66
  keywords_task,
@@ -71,6 +73,7 @@ async def main():
71
73
  summary_task,
72
74
  translation_task,
73
75
  propositionize_task,
76
+ check_fact_task,
74
77
  )
75
78
 
76
79
  for tool_output in (
@@ -85,6 +88,7 @@ async def main():
85
88
  summary,
86
89
  translation,
87
90
  propositionize,
91
+ check_fact,
88
92
  ):
89
93
  print(repr(tool_output))
90
94
 
@@ -96,6 +96,13 @@ propositionize = t.propositionize(
96
96
  )
97
97
  print(repr(propositionize))
98
98
 
99
+ # check_fact
100
+ check_fact = t.check_fact(
101
+ text="امام نهم در ایران به خاک سپرده شد",
102
+ source_text="حرم مطهر امام رضا علیه السلام در مشهد مقدس هست",
103
+ )
104
+ print(repr(check_fact))
105
+
99
106
 
100
107
  # Custom tool
101
108
  class Student(BaseModel):
@@ -44,16 +44,44 @@ class PromptLoader:
44
44
  if self.MAIN_TEMPLATE not in data:
45
45
  raise PromptError(f"Missing 'main_template' in {prompt_file}")
46
46
 
47
+ if self.ANALYZE_TEMPLATE not in data:
48
+ raise PromptError(f"Missing 'analyze_template' in {prompt_file}")
49
+
47
50
  if mode and mode not in data.get(self.MAIN_TEMPLATE, {}):
48
51
  raise PromptError(f"Mode '{mode}' not found in {prompt_file}")
49
52
 
50
- return {
51
- self.MAIN_TEMPLATE: data[self.MAIN_TEMPLATE][mode]
53
+ # Extract templates based on mode
54
+ main_template = (
55
+ data[self.MAIN_TEMPLATE][mode]
52
56
  if mode and isinstance(data[self.MAIN_TEMPLATE], dict)
53
- else data[self.MAIN_TEMPLATE],
54
- self.ANALYZE_TEMPLATE: data.get(self.ANALYZE_TEMPLATE, {}).get(mode)
55
- if mode and isinstance(data.get(self.ANALYZE_TEMPLATE), dict)
56
- else data.get(self.ANALYZE_TEMPLATE, ""),
57
+ else data[self.MAIN_TEMPLATE]
58
+ )
59
+
60
+ analyze_template = (
61
+ data[self.ANALYZE_TEMPLATE][mode]
62
+ if mode and isinstance(data[self.ANALYZE_TEMPLATE], dict)
63
+ else data[self.ANALYZE_TEMPLATE]
64
+ )
65
+
66
+ if not main_template or not main_template.strip():
67
+ raise PromptError(
68
+ f"Empty main_template in {prompt_file}"
69
+ + (f" for mode '{mode}'" if mode else "")
70
+ )
71
+
72
+ if (
73
+ not analyze_template
74
+ or not analyze_template.strip()
75
+ or analyze_template.strip() in ["{analyze_template}", "{}"]
76
+ ):
77
+ raise PromptError(
78
+ "analyze_template cannot be empty"
79
+ + (f" for mode '{mode}'" if mode else "")
80
+ )
81
+
82
+ return {
83
+ self.MAIN_TEMPLATE: main_template,
84
+ self.ANALYZE_TEMPLATE: analyze_template,
57
85
  }
58
86
 
59
87
  except yaml.YAMLError as e:
@@ -0,0 +1,19 @@
1
+ main_template: |
2
+ You are an expert in determining whether a statement can be concluded from the source text or not.
3
+ You must return a boolean value: True or False.
4
+ Return True if the statement can be concluded from the source, and False otherwise.
5
+ Respond only in JSON format (Output should be a boolean):
6
+ {{"result": True/False}}
7
+ The statement is:
8
+ {input}
9
+ The source text is:
10
+ {source_text}
11
+
12
+ analyze_template: |
13
+ You should analyze a statement and a source text and provide a brief,
14
+ summarized analysis that could help in determining that can the statement
15
+ be concluded from the source or not.
16
+ The statement is:
17
+ {input}
18
+ The source text is:
19
+ {source_text}
@@ -0,0 +1,22 @@
1
+ main_template: |
2
+ You are an expert data analyst specializing in Information Extraction.
3
+ Your task is to extract a list of "Atomic Propositions" from the provided text.
4
+
5
+ Definition of Atomic Proposition:
6
+ A single, self-contained statement of fact that is concise and verifiable.
7
+
8
+ Strict Guidelines:
9
+ 1. Remove Meta-Data: STRICTLY EXCLUDE all citations, references, URLs, source attributions (e.g., "Source: makarem.ir"), and conversational fillers (e.g., "Based on the documents...", "In conclusion...").
10
+ 2. Resolve Context: Replace pronouns ("it", "this", "they") with the specific nouns they refer to. Each proposition must make sense in isolation.
11
+ 3. Preserve Logic: Keep conditions attached to their facts. Do not split a rule from its condition (e.g., "If X, then Y" should be one proposition).
12
+ 4. No Redundancy: Do not extract summary statements that merely repeat facts already listed.
13
+
14
+ Extract the atomic propositions from the following text:
15
+ {input}
16
+
17
+ analyze_template: |
18
+ We want to analyze this text snippet and think about where we can split sentence to atomic meaningful propositions.
19
+ An atomic proposition is a single, self-contained fact that is concise,
20
+ verifiable, and does not rely on external context.
21
+ You just have to think around the possible propositions in the text and how a proposition can be made.
22
+ {input}
@@ -4,4 +4,4 @@ main_template: |
4
4
  {output_model_str}
5
5
 
6
6
  analyze_template: |
7
-
7
+ {analyze_template}
@@ -44,6 +44,8 @@ class AsyncTheTool:
44
44
  """
45
45
  Categorize a text into a category / category tree.
46
46
 
47
+ Important Note: category_tree mode is EXPERIMENTAL, you can use it but it isn't reliable.
48
+
47
49
  Arguments:
48
50
  text: The input text to categorize
49
51
  categories: The category / category_tree to give to LLM
@@ -822,6 +824,8 @@ class AsyncTheTool:
822
824
  """
823
825
  Translate text between languages.
824
826
 
827
+ Important Note: This tool is EXPERIMENTAL, you can use it but it isn't reliable.
828
+
825
829
  Arguments:
826
830
  text: The input text to translate
827
831
  target_language: The target language for translation
@@ -883,7 +887,7 @@ class AsyncTheTool:
883
887
 
884
888
  return output
885
889
 
886
- async def detect_entity(
890
+ async def propositionize(
887
891
  self,
888
892
  text: str,
889
893
  with_analysis: bool = False,
@@ -897,7 +901,9 @@ class AsyncTheTool:
897
901
  priority: int | None = 0,
898
902
  ) -> Models.ToolOutput:
899
903
  """
900
- Detects entities in a given text based on the entity_detector.yaml prompt.
904
+ Proposition input text to meaningful sentences.
905
+
906
+ Important Note: This tool is EXPERIMENTAL, you can use it but it isn't reliable.
901
907
 
902
908
  Arguments:
903
909
  text: The input text
@@ -913,7 +919,7 @@ class AsyncTheTool:
913
919
 
914
920
  Returns:
915
921
  ToolOutput: Object containing:
916
- - result (list[Entity]): The entities
922
+ - result (list[str]): The propositions
917
923
  - logprobs (list | None): Probability data if logprobs enabled
918
924
  - analysis (str | None): Detailed reasoning if with_analysis enabled
919
925
  - process (str | None): Description of the process used
@@ -938,8 +944,8 @@ class AsyncTheTool:
938
944
  max_validation_retries=max_validation_retries,
939
945
  priority=priority,
940
946
  # Internal parameters
941
- prompt_file="detect_entity.yaml",
942
- output_model=Models.EntityDetectorOutput,
947
+ prompt_file="propositionize.yaml",
948
+ output_model=Models.ListStrOutput,
943
949
  mode=None,
944
950
  )
945
951
  end = datetime.now()
@@ -959,9 +965,10 @@ class AsyncTheTool:
959
965
 
960
966
  return output
961
967
 
962
- async def propositionize(
968
+ async def check_fact(
963
969
  self,
964
970
  text: str,
971
+ source_text: str,
965
972
  with_analysis: bool = False,
966
973
  output_lang: str | None = None,
967
974
  user_prompt: str | None = None,
@@ -973,10 +980,13 @@ class AsyncTheTool:
973
980
  priority: int | None = 0,
974
981
  ) -> Models.ToolOutput:
975
982
  """
976
- Proposition input text to meaningful sentences.
983
+ Checks wheather a statement is relevant to the source text or not.
984
+
985
+ Important Note: This tool is EXPERIMENTAL, you can use it but it isn't reliable.
977
986
 
978
987
  Arguments:
979
988
  text: The input text
989
+ source_text: the source text that we want to check relation of text to it
980
990
  with_analysis: Whether to include detailed reasoning analysis
981
991
  output_lang: Language for the output summary
982
992
  user_prompt: Additional instructions for summarization
@@ -989,7 +999,7 @@ class AsyncTheTool:
989
999
 
990
1000
  Returns:
991
1001
  ToolOutput: Object containing:
992
- - result (list[str]): The propositions
1002
+ - result (bool): statement is relevant to source text or not
993
1003
  - logprobs (list | None): Probability data if logprobs enabled
994
1004
  - analysis (str | None): Detailed reasoning if with_analysis enabled
995
1005
  - process (str | None): Description of the process used
@@ -1014,14 +1024,14 @@ class AsyncTheTool:
1014
1024
  max_validation_retries=max_validation_retries,
1015
1025
  priority=priority,
1016
1026
  # Internal parameters
1017
- prompt_file="propositionize.yaml",
1018
- output_model=Models.ListStrOutput,
1027
+ prompt_file="check_fact.yaml",
1028
+ output_model=Models.BoolOutput,
1019
1029
  mode=None,
1030
+ source_text=source_text,
1020
1031
  )
1021
1032
  end = datetime.now()
1022
1033
  output.execution_time = (end - start).total_seconds()
1023
1034
  return output
1024
-
1025
1035
  except PromptError as e:
1026
1036
  output.errors.append(f"Prompt error: {e}")
1027
1037
  except LLMError as e:
@@ -1039,6 +1049,8 @@ class AsyncTheTool:
1039
1049
  self,
1040
1050
  prompt: str,
1041
1051
  output_model: Any,
1052
+ with_analysis: bool = False,
1053
+ analyze_template: str | None = None,
1042
1054
  output_lang: str | None = None,
1043
1055
  temperature: float | None = None,
1044
1056
  logprobs: bool | None = None,
@@ -1050,8 +1062,13 @@ class AsyncTheTool:
1050
1062
  """
1051
1063
  Custom tool that can do almost anything!
1052
1064
 
1065
+ Important Note: This tool is EXPERIMENTAL, you can use it but it isn't reliable.
1066
+
1053
1067
  Arguments:
1054
- text: The user prompt
1068
+ prompt: The user prompt
1069
+ output_model: Pydantic BaseModel used for structured output
1070
+ with_analysis: Whether to include detailed reasoning analysis
1071
+ analyze_template: The analyze template used for reasoning analysis
1055
1072
  output_lang: Language for the output summary
1056
1073
  temperature: Controls randomness (0.0 = deterministic, 1.0 = creative)
1057
1074
  logprobs: Whether to return token probability information
@@ -1078,6 +1095,8 @@ class AsyncTheTool:
1078
1095
  # User paramaeters
1079
1096
  text=prompt,
1080
1097
  output_model=output_model,
1098
+ with_analysis=with_analysis,
1099
+ analyze_template=analyze_template,
1081
1100
  output_model_str=output_model.model_json_schema(),
1082
1101
  output_lang=output_lang,
1083
1102
  temperature=temperature,
@@ -1089,7 +1108,6 @@ class AsyncTheTool:
1089
1108
  # Internal parameters
1090
1109
  prompt_file="run_custom.yaml",
1091
1110
  user_prompt=None,
1092
- with_analysis=False,
1093
1111
  mode=None,
1094
1112
  )
1095
1113
  end = datetime.now()
@@ -44,6 +44,8 @@ class TheTool:
44
44
  """
45
45
  Categorize a text into a category / category tree.
46
46
 
47
+ Important Note: category_tree mode is EXPERIMENTAL, you can use it but it isn't reliable.
48
+
47
49
  Arguments:
48
50
  text: The input text to categorize
49
51
  categories: The category / category_tree to give to LLM
@@ -822,6 +824,8 @@ class TheTool:
822
824
  """
823
825
  Translate text between languages.
824
826
 
827
+ Important Note: This tool is EXPERIMENTAL, you can use it but it isn't reliable.
828
+
825
829
  Arguments:
826
830
  text: The input text to translate
827
831
  target_language: The target language for translation
@@ -883,7 +887,7 @@ class TheTool:
883
887
 
884
888
  return output
885
889
 
886
- def detect_entity(
890
+ def propositionize(
887
891
  self,
888
892
  text: str,
889
893
  with_analysis: bool = False,
@@ -897,7 +901,9 @@ class TheTool:
897
901
  priority: int | None = 0,
898
902
  ) -> Models.ToolOutput:
899
903
  """
900
- Detects entities in a given text based on the entity_detector.yaml prompt.
904
+ Proposition input text to meaningful sentences.
905
+
906
+ Important Note: This tool is EXPERIMENTAL, you can use it but it isn't reliable.
901
907
 
902
908
  Arguments:
903
909
  text: The input text
@@ -913,7 +919,7 @@ class TheTool:
913
919
 
914
920
  Returns:
915
921
  ToolOutput: Object containing:
916
- - result (list[Entity]): The entities
922
+ - result (list[str]): The propositions
917
923
  - logprobs (list | None): Probability data if logprobs enabled
918
924
  - analysis (str | None): Detailed reasoning if with_analysis enabled
919
925
  - process (str | None): Description of the process used
@@ -938,8 +944,8 @@ class TheTool:
938
944
  max_validation_retries=max_validation_retries,
939
945
  priority=priority,
940
946
  # Internal parameters
941
- prompt_file="detect_entity.yaml",
942
- output_model=Models.EntityDetectorOutput,
947
+ prompt_file="propositionize.yaml",
948
+ output_model=Models.ListStrOutput,
943
949
  mode=None,
944
950
  )
945
951
  end = datetime.now()
@@ -959,9 +965,10 @@ class TheTool:
959
965
 
960
966
  return output
961
967
 
962
- def propositionize(
968
+ def check_fact(
963
969
  self,
964
970
  text: str,
971
+ source_text: str,
965
972
  with_analysis: bool = False,
966
973
  output_lang: str | None = None,
967
974
  user_prompt: str | None = None,
@@ -973,10 +980,13 @@ class TheTool:
973
980
  priority: int | None = 0,
974
981
  ) -> Models.ToolOutput:
975
982
  """
976
- Proposition input text to meaningful sentences.
983
+ Checks wheather a statement is relevant to the source text or not.
984
+
985
+ Important Note: This tool is EXPERIMENTAL, you can use it but it isn't reliable.
977
986
 
978
987
  Arguments:
979
988
  text: The input text
989
+ source_text: the source text that we want to check relation of text to it
980
990
  with_analysis: Whether to include detailed reasoning analysis
981
991
  output_lang: Language for the output summary
982
992
  user_prompt: Additional instructions for summarization
@@ -989,7 +999,7 @@ class TheTool:
989
999
 
990
1000
  Returns:
991
1001
  ToolOutput: Object containing:
992
- - result (list[str]): The propositions
1002
+ - result (bool): statement is relevant to source text or not
993
1003
  - logprobs (list | None): Probability data if logprobs enabled
994
1004
  - analysis (str | None): Detailed reasoning if with_analysis enabled
995
1005
  - process (str | None): Description of the process used
@@ -998,7 +1008,6 @@ class TheTool:
998
1008
  - errors (list(str) | None): Errors occured during tool call
999
1009
  """
1000
1010
  output = Models.ToolOutput()
1001
-
1002
1011
  try:
1003
1012
  start = datetime.now()
1004
1013
  output = self._operator.run(
@@ -1014,9 +1023,10 @@ class TheTool:
1014
1023
  max_validation_retries=max_validation_retries,
1015
1024
  priority=priority,
1016
1025
  # Internal parameters
1017
- prompt_file="propositionize.yaml",
1018
- output_model=Models.ListStrOutput,
1026
+ prompt_file="check_fact.yaml",
1027
+ output_model=Models.BoolOutput,
1019
1028
  mode=None,
1029
+ source_text=source_text,
1020
1030
  )
1021
1031
  end = datetime.now()
1022
1032
  output.execution_time = (end - start).total_seconds()
@@ -1039,6 +1049,8 @@ class TheTool:
1039
1049
  self,
1040
1050
  prompt: str,
1041
1051
  output_model: Any,
1052
+ with_analysis: bool = False,
1053
+ analyze_template: str | None = None,
1042
1054
  output_lang: str | None = None,
1043
1055
  temperature: float | None = None,
1044
1056
  logprobs: bool | None = None,
@@ -1050,8 +1062,13 @@ class TheTool:
1050
1062
  """
1051
1063
  Custom tool that can do almost anything!
1052
1064
 
1065
+ Important Note: This tool is EXPERIMENTAL, you can use it but it isn't reliable.
1066
+
1053
1067
  Arguments:
1054
- text: The user prompt
1068
+ prompt: The user prompt
1069
+ output_model: Pydantic BaseModel used for structured output
1070
+ with_analysis: Whether to include detailed reasoning analysis
1071
+ analyze_template: The analyze template used for reasoning analysis
1055
1072
  output_lang: Language for the output summary
1056
1073
  temperature: Controls randomness (0.0 = deterministic, 1.0 = creative)
1057
1074
  logprobs: Whether to return token probability information
@@ -1078,6 +1095,8 @@ class TheTool:
1078
1095
  # User paramaeters
1079
1096
  text=prompt,
1080
1097
  output_model=output_model,
1098
+ with_analysis=with_analysis,
1099
+ analyze_template=analyze_template,
1081
1100
  output_model_str=output_model.model_json_schema(),
1082
1101
  output_lang=output_lang,
1083
1102
  temperature=temperature,
@@ -1089,7 +1108,6 @@ class TheTool:
1089
1108
  # Internal parameters
1090
1109
  prompt_file="run_custom.yaml",
1091
1110
  user_prompt=None,
1092
- with_analysis=False,
1093
1111
  mode=None,
1094
1112
  )
1095
1113
  end = datetime.now()
@@ -1,22 +0,0 @@
1
- main_template: |
2
- You are an expert Named Entity Recognition (NER) system. Extract entities from the text.
3
- The output must strictly follow the provided Pydantic schema.
4
-
5
- Mapping Rule:
6
- - Person: شخص
7
- - Location: مکان
8
- - Time: زمان
9
- - Living Beings: موجود زنده
10
- - Organization: سازمان
11
- - Concept: مفهوم
12
-
13
- CRITICAL:
14
- 1. The final output structure must be a complete JSON object matching the Pydantic schema (List[Entity]).
15
- 2. Both the extracted text and the type must be in Persian, using the exact mapping provided above.
16
-
17
- Here is the text: {input}
18
-
19
- analyze_template: |
20
- Analyze the following text to identify all potential named entities and their categories (Person, Location, Time, Living Beings, Organization, Concept).
21
- Provide a brief summary of the entities identified that will help the main process to extract them accurately and apply the correct Persian type label.
22
- Here is the text: {input}
@@ -1,15 +0,0 @@
1
- main_template: |
2
- You are an expert in breaking down text into atomic propositions in that language.
3
- An atomic proposition is a single, self-contained fact that is concise, verifiable,
4
- and does not rely on external context.
5
- Each proposition must stand alone.
6
- Rewrite sentences if needed to keep the context saved in each sentence.
7
- Extract the atomic propositions of this text:
8
- {input}
9
-
10
- analyze_template: |
11
- We want to analyze this text snippet and think about where we can split sentence to atomic meaningful propositions.
12
- An atomic proposition is a single, self-contained fact that is concise,
13
- verifiable, and does not rely on external context.
14
- You just have to think around the possible propositions in the text and how a proposition can be made.
15
- {input}