versionhq 1.2.1.1__tar.gz → 1.2.1.3__tar.gz

{versionhq-1.2.1.1 → versionhq-1.2.1.3}/.github/workflows/deploy_docs.yml

@@ -26,5 +26,5 @@ jobs:
           path: .cache
           restore-keys: |
             mkdocs-material-
-      - run: pip install mkdocs-material 
-      - run: mkdocs gh-deploy --force
+      - run: pip install mkdocs-material
+      - run: mkdocs gh-deploy --force --clean

{versionhq-1.2.1.1 → versionhq-1.2.1.3}/.gitignore

@@ -7,24 +7,28 @@ _memo
 _idea.py
 
 
-
 dist/
 lib/
 build/
 .pypirc
+*egg-info/
+
 uploads/
 sample_dataset/
 chroma.sqlite3
-*egg-info/
+
 test/
 db/
 rc-tests/*
 temp/*
 .vscode/*
 assets/*
+
 memo.txt
 site/
 
+_logs/
+.cache
 __pycache__
 .ruff_cache/
 .pytest_cache/

{versionhq-1.2.1.1 → versionhq-1.2.1.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.2
 Name: versionhq
-Version: 1.2.1.1
+Version: 1.2.1.3
 Summary: An agentic orchestration framework for building agent networks that handle task automation.
 Author-email: Kuriko Iwai <kuriko@versi0n.io>
 License: MIT License
@@ -89,7 +89,7 @@ Requires-Dist: pygraphviz>=1.14; extra == "pygraphviz"
 ![pyenv ver](https://img.shields.io/badge/pyenv-2.5.0-orange)
 
 
-A Python framework for agentic orchestration that handles complex task automation without human interaction.
+Agentic orchestration framework for multi-agent networks and task graph for complex task automation.
 
 **Visit:**
 
@@ -115,13 +115,12 @@ A Python framework for agentic orchestration that handles complex task automatio
   - [Forming a agent network](#forming-a-agent-network)
   - [Executing tasks](#executing-tasks)
   - [Supervising](#supervising)
-- [Supervising](#supervising-1)
 - [Technologies Used](#technologies-used)
 - [Project Structure](#project-structure)
-- [Setting Up a Project](#setting-up-a-project)
-  - [1. Installing package manager](#1-installing-package-manager)
-  - [2. Installing dependencies](#2-installing-dependencies)
-  - [3. Adding env secrets to .env file](#3-adding-env-secrets-to-env-file)
+- [Setting Up Your Project](#setting-up-your-project)
+  - [Installing package manager](#installing-package-manager)
+  - [Installing dependencies](#installing-dependencies)
+  - [Adding env secrets to .env file](#adding-env-secrets-to-env-file)
 - [Contributing](#contributing)
   - [Steps](#steps)
   - [Package Management with uv](#package-management-with-uv)
@@ -189,7 +188,14 @@ task_graph.add_dependency(
    type=vhq.DependencyType.FINISH_TO_FINISH, lag=1, required=False, weight=3
 )
 
+# To visualize the graph:
 task_graph.visualize()
+
+# To start executing nodes:
+latest_output, outputs = task_graph.activate()
+
+assert isinstance(last_task_output, vhq.TaskOutput)
+assert [k in task_graph.nodes.keys() and v and isinstance(v, vhq.TaskOutput) for k, v in outputs.items()]
 ```
 
 <hr />
@@ -257,7 +263,7 @@ agent.update(
    res = network.launch()
    ```
 
-   This will form a network with multiple agents on `Formation` and return `TaskOutput` object with output in JSON, plane text, Pydantic model format with evaluation.
+This will form a network with multiple agents on `Formation` and return `TaskOutput` object with output in JSON, plane text, Pydantic model format with evaluation.
 
 
 ### Executing tasks
@@ -267,47 +273,45 @@ You can simply build an agent using `Agent` model and execute the task using `Ta
 By default, agents prioritize JSON over plane text outputs.
 
 
-   ```python
-   import versionhq as vhq
-   from pydantic import BaseModel
+```python
+import versionhq as vhq
+from pydantic import BaseModel
 
-   class CustomOutput(BaseModel):
-      test1: str
-      test2: list[str]
+class CustomOutput(BaseModel):
+   test1: str
+   test2: list[str]
 
-   def dummy_func(message: str, test1: str, test2: list[str]) -> str:
-      return f"""{message}: {test1}, {", ".join(test2)}"""
+def dummy_func(message: str, test1: str, test2: list[str]) -> str:
+   return f"""{message}: {test1}, {", ".join(test2)}"""
 
-   task = vhq.Task(
-      description="Amazing task",
-      pydantic_output=CustomOutput,
-      callback=dummy_func,
-      callback_kwargs=dict(message="Hi! Here is the result: ")
-   )
+task = vhq.Task(
+   description="Amazing task",
+   pydantic_output=CustomOutput,
+   callback=dummy_func,
+   callback_kwargs=dict(message="Hi! Here is the result: ")
+)
 
-   res = task.execute(context="amazing context to consider.")
-   print(res)
-   ```
+res = task.execute(context="amazing context to consider.")
+print(res)
+```
 
 
 This will return a `TaskOutput` object that stores response in plane text, JSON, and Pydantic model: `CustomOutput` formats with a callback result, tool output (if given), and evaluation results (if given).
 
-   ```python
-   res == TaskOutput(
-      task_id=UUID('<TASK UUID>'),
-      raw='{\"test1\":\"random str\", \"test2\":[\"str item 1\", \"str item 2\", \"str item 3\"]}',
-      json_dict={'test1': 'random str', 'test2': ['str item 1', 'str item 2', 'str item 3']},
-      pydantic=<class '__main__.CustomOutput'>,
-      tool_output=None,
-      callback_output='Hi! Here is the result: random str, str item 1, str item 2, str item 3', # returned a plain text summary
-      evaluation=None
-   )
-   ```
+```python
+res == TaskOutput(
+   task_id=UUID('<TASK UUID>'),
+   raw='{\"test1\":\"random str\", \"test2\":[\"str item 1\", \"str item 2\", \"str item 3\"]}',
+   json_dict={'test1': 'random str', 'test2': ['str item 1', 'str item 2', 'str item 3']},
+   pydantic=<class '__main__.CustomOutput'>,
+   tool_output=None,
+   callback_output='Hi! Here is the result: random str, str item 1, str item 2, str item 3', # returned a plain text summary
+   evaluation=None
+)
+```
 
 ### Supervising
 
-## Supervising
-
 To create an agent network with one or more manager agents, designate members using the `is_manager` tag.
 
 ```python
@@ -349,34 +353,34 @@ Tasks can be delegated to a manager, peers within the agent network, or a comple
 
 ## Technologies Used
 
-**Task Graph**
-
-* [NetworkX](https://networkx.org/documentation/stable/reference/introduction.html): A Python package to analyze, create, and manipulate complex graph networks.
-* [Matplotlib](https://matplotlib.org/stable/index.html): Visualization library
-* [Graphviz](https://graphviz.org/about/): Graph visualization software
-
-
 **Schema, Data Validation**
 
 * [Pydantic](https://docs.pydantic.dev/latest/): Data validation and serialization library for Python.
 * [Upstage](https://console.upstage.ai/docs/getting-started/overview): Document processer for ML tasks. (Use `Document Parser API` to extract data from documents)
 * [Docling](https://ds4sd.github.io/docling/): Document parsing
 
+**Workflow, Task Graph**
 
-**Storage**
-
-* [mem0ai](https://docs.mem0.ai/quickstart#install-package): Agents' memory storage and management.
-* [Chroma DB](https://docs.trychroma.com/): Vector database for storing and querying usage data.
-* [SQLite](https://www.sqlite.org/docs.html): C-language library to implements a small SQL database engine.
+* [NetworkX](https://networkx.org/documentation/stable/reference/introduction.html): A Python package to analyze, create, and manipulate complex graph networks.
+* [Matplotlib](https://matplotlib.org/stable/index.html): For graph visualization.
+* [Graphviz](https://graphviz.org/about/): For graph visualization.
 
-**LLM Integration**
+**LLM Curation**
 
-* [LiteLLM](https://docs.litellm.ai/docs/providers): Integration to diverse LLMs
+* [LiteLLM](https://docs.litellm.ai/docs/providers): LLM orchestration platform
 
 **Tools**
 
 * [Composio](https://composio.dev/): Conect RAG agents with external tools, Apps, and APIs to perform actions and receive triggers. We use [tools](https://composio.dev/tools) and [RAG tools](https://app.composio.dev/app/ragtool) from Composio toolset.
 
+
+**Storage**
+
+* [mem0ai](https://docs.mem0.ai/quickstart#install-package): Agents' memory storage and management.
+* [Chroma DB](https://docs.trychroma.com/): Vector database for storing and querying usage data.
+* [SQLite](https://www.sqlite.org/docs.html): C-language library to implements a small SQL database engine.
+
+
 **Deployment**
 
 * **Python**: Primary programming language. v3.12.x is recommended
@@ -410,16 +414,19 @@ src/
 │     └── ...
 │
 └── uploads/  [.gitignore]    # Local directory to store uploaded files such as graphviz diagrams generatd by `Network` class
-|
+│
+└── _logs/    [.gitignore]    # Local directory to store error/warning logs for debugging
+│
+│
 pyproject.toml                # Project config
 
 ```
 
 <hr />
 
-## Setting Up a Project
+## Setting Up Your Project
 
-### 1. Installing package manager
+### Installing package manager
 
    For MacOS:
 
@@ -433,7 +440,7 @@ pyproject.toml                # Project config
    ```
 
 
-### 2. Installing dependencies
+### Installing dependencies
 
    ```
    uv venv
@@ -466,7 +473,7 @@ pyproject.toml                # Project config
 
    - `torch`/`Docling` related errors: Set up default Python version either `3.11.x` or `3.12.x` (same as AssertionError)
 
-### 3. Adding env secrets to .env file
+### Adding env secrets to .env file
 
 Create `.env` file in the project root and add following:
 
@@ -479,6 +486,7 @@ Create `.env` file in the project root and add following:
    [OTHER_LLM_INTERFACE_PROVIDER_OF_YOUR_CHOICE]_API_KEY=your-api-key
    ```
 
+
 <hr />
 
 ## Contributing
@@ -583,7 +591,7 @@ Common issues and solutions:
 
 * Issues related to dependencies: `rm -rf uv.lock`, `uv cache clean`, `uv venv`, and run `uv pip install -r requirements.txt -v`.
 
-* Issues related to the AI agents or RAG system: Check the `output.log` file for detailed error messages and stack traces.
+* Issues related to agents and other systems: Check `_logs` directory for detailed error messages and stack traces.
 
 * Issues related to `Python quit unexpectedly`: Check [this stackoverflow article](https://stackoverflow.com/questions/59888499/macos-catalina-python-quit-unexpectedly-error).
 

{versionhq-1.2.1.1 → versionhq-1.2.1.3}/README.md

@@ -8,7 +8,7 @@
 ![pyenv ver](https://img.shields.io/badge/pyenv-2.5.0-orange)
 
 
-A Python framework for agentic orchestration that handles complex task automation without human interaction.
+Agentic orchestration framework for multi-agent networks and task graph for complex task automation.
 
 **Visit:**
 
@@ -34,13 +34,12 @@ A Python framework for agentic orchestration that handles complex task automatio
   - [Forming a agent network](#forming-a-agent-network)
   - [Executing tasks](#executing-tasks)
   - [Supervising](#supervising)
-- [Supervising](#supervising-1)
 - [Technologies Used](#technologies-used)
 - [Project Structure](#project-structure)
-- [Setting Up a Project](#setting-up-a-project)
-  - [1. Installing package manager](#1-installing-package-manager)
-  - [2. Installing dependencies](#2-installing-dependencies)
-  - [3. Adding env secrets to .env file](#3-adding-env-secrets-to-env-file)
+- [Setting Up Your Project](#setting-up-your-project)
+  - [Installing package manager](#installing-package-manager)
+  - [Installing dependencies](#installing-dependencies)
+  - [Adding env secrets to .env file](#adding-env-secrets-to-env-file)
 - [Contributing](#contributing)
   - [Steps](#steps)
   - [Package Management with uv](#package-management-with-uv)
@@ -108,7 +107,14 @@ task_graph.add_dependency(
    type=vhq.DependencyType.FINISH_TO_FINISH, lag=1, required=False, weight=3
 )
 
+# To visualize the graph:
 task_graph.visualize()
+
+# To start executing nodes:
+latest_output, outputs = task_graph.activate()
+
+assert isinstance(last_task_output, vhq.TaskOutput)
+assert [k in task_graph.nodes.keys() and v and isinstance(v, vhq.TaskOutput) for k, v in outputs.items()]
 ```
 
 <hr />
@@ -176,7 +182,7 @@ agent.update(
    res = network.launch()
    ```
 
-   This will form a network with multiple agents on `Formation` and return `TaskOutput` object with output in JSON, plane text, Pydantic model format with evaluation.
+This will form a network with multiple agents on `Formation` and return `TaskOutput` object with output in JSON, plane text, Pydantic model format with evaluation.
 
 
 ### Executing tasks
@@ -186,47 +192,45 @@ You can simply build an agent using `Agent` model and execute the task using `Ta
 By default, agents prioritize JSON over plane text outputs.
 
 
-   ```python
-   import versionhq as vhq
-   from pydantic import BaseModel
+```python
+import versionhq as vhq
+from pydantic import BaseModel
 
-   class CustomOutput(BaseModel):
-      test1: str
-      test2: list[str]
+class CustomOutput(BaseModel):
+   test1: str
+   test2: list[str]
 
-   def dummy_func(message: str, test1: str, test2: list[str]) -> str:
-      return f"""{message}: {test1}, {", ".join(test2)}"""
+def dummy_func(message: str, test1: str, test2: list[str]) -> str:
+   return f"""{message}: {test1}, {", ".join(test2)}"""
 
-   task = vhq.Task(
-      description="Amazing task",
-      pydantic_output=CustomOutput,
-      callback=dummy_func,
-      callback_kwargs=dict(message="Hi! Here is the result: ")
-   )
+task = vhq.Task(
+   description="Amazing task",
+   pydantic_output=CustomOutput,
+   callback=dummy_func,
+   callback_kwargs=dict(message="Hi! Here is the result: ")
+)
 
-   res = task.execute(context="amazing context to consider.")
-   print(res)
-   ```
+res = task.execute(context="amazing context to consider.")
+print(res)
+```
 
 
 This will return a `TaskOutput` object that stores response in plane text, JSON, and Pydantic model: `CustomOutput` formats with a callback result, tool output (if given), and evaluation results (if given).
 
-   ```python
-   res == TaskOutput(
-      task_id=UUID('<TASK UUID>'),
-      raw='{\"test1\":\"random str\", \"test2\":[\"str item 1\", \"str item 2\", \"str item 3\"]}',
-      json_dict={'test1': 'random str', 'test2': ['str item 1', 'str item 2', 'str item 3']},
-      pydantic=<class '__main__.CustomOutput'>,
-      tool_output=None,
-      callback_output='Hi! Here is the result: random str, str item 1, str item 2, str item 3', # returned a plain text summary
-      evaluation=None
-   )
-   ```
+```python
+res == TaskOutput(
+   task_id=UUID('<TASK UUID>'),
+   raw='{\"test1\":\"random str\", \"test2\":[\"str item 1\", \"str item 2\", \"str item 3\"]}',
+   json_dict={'test1': 'random str', 'test2': ['str item 1', 'str item 2', 'str item 3']},
+   pydantic=<class '__main__.CustomOutput'>,
+   tool_output=None,
+   callback_output='Hi! Here is the result: random str, str item 1, str item 2, str item 3', # returned a plain text summary
+   evaluation=None
+)
+```
 
 ### Supervising
 
-## Supervising
-
 To create an agent network with one or more manager agents, designate members using the `is_manager` tag.
 
 ```python
@@ -268,34 +272,34 @@ Tasks can be delegated to a manager, peers within the agent network, or a comple
 
 ## Technologies Used
 
-**Task Graph**
-
-* [NetworkX](https://networkx.org/documentation/stable/reference/introduction.html): A Python package to analyze, create, and manipulate complex graph networks.
-* [Matplotlib](https://matplotlib.org/stable/index.html): Visualization library
-* [Graphviz](https://graphviz.org/about/): Graph visualization software
-
-
 **Schema, Data Validation**
 
 * [Pydantic](https://docs.pydantic.dev/latest/): Data validation and serialization library for Python.
 * [Upstage](https://console.upstage.ai/docs/getting-started/overview): Document processer for ML tasks. (Use `Document Parser API` to extract data from documents)
 * [Docling](https://ds4sd.github.io/docling/): Document parsing
 
+**Workflow, Task Graph**
 
-**Storage**
-
-* [mem0ai](https://docs.mem0.ai/quickstart#install-package): Agents' memory storage and management.
-* [Chroma DB](https://docs.trychroma.com/): Vector database for storing and querying usage data.
-* [SQLite](https://www.sqlite.org/docs.html): C-language library to implements a small SQL database engine.
+* [NetworkX](https://networkx.org/documentation/stable/reference/introduction.html): A Python package to analyze, create, and manipulate complex graph networks.
+* [Matplotlib](https://matplotlib.org/stable/index.html): For graph visualization.
+* [Graphviz](https://graphviz.org/about/): For graph visualization.
 
-**LLM Integration**
+**LLM Curation**
 
-* [LiteLLM](https://docs.litellm.ai/docs/providers): Integration to diverse LLMs
+* [LiteLLM](https://docs.litellm.ai/docs/providers): LLM orchestration platform
 
 **Tools**
 
 * [Composio](https://composio.dev/): Conect RAG agents with external tools, Apps, and APIs to perform actions and receive triggers. We use [tools](https://composio.dev/tools) and [RAG tools](https://app.composio.dev/app/ragtool) from Composio toolset.
 
+
+**Storage**
+
+* [mem0ai](https://docs.mem0.ai/quickstart#install-package): Agents' memory storage and management.
+* [Chroma DB](https://docs.trychroma.com/): Vector database for storing and querying usage data.
+* [SQLite](https://www.sqlite.org/docs.html): C-language library to implements a small SQL database engine.
+
+
 **Deployment**
 
 * **Python**: Primary programming language. v3.12.x is recommended
@@ -329,16 +333,19 @@ src/
 │     └── ...
 │
 └── uploads/  [.gitignore]    # Local directory to store uploaded files such as graphviz diagrams generatd by `Network` class
-|
+│
+└── _logs/    [.gitignore]    # Local directory to store error/warning logs for debugging
+│
+│
 pyproject.toml                # Project config
 
 ```
 
 <hr />
 
-## Setting Up a Project
+## Setting Up Your Project
 
-### 1. Installing package manager
+### Installing package manager
 
    For MacOS:
 
@@ -352,7 +359,7 @@ pyproject.toml                # Project config
    ```
 
 
-### 2. Installing dependencies
+### Installing dependencies
 
    ```
    uv venv
@@ -385,7 +392,7 @@ pyproject.toml                # Project config
 
    - `torch`/`Docling` related errors: Set up default Python version either `3.11.x` or `3.12.x` (same as AssertionError)
 
-### 3. Adding env secrets to .env file
+### Adding env secrets to .env file
 
 Create `.env` file in the project root and add following:
 
@@ -398,6 +405,7 @@ Create `.env` file in the project root and add following:
    [OTHER_LLM_INTERFACE_PROVIDER_OF_YOUR_CHOICE]_API_KEY=your-api-key
    ```
 
+
 <hr />
 
 ## Contributing
@@ -502,7 +510,7 @@ Common issues and solutions:
 
 * Issues related to dependencies: `rm -rf uv.lock`, `uv cache clean`, `uv venv`, and run `uv pip install -r requirements.txt -v`.
 
-* Issues related to the AI agents or RAG system: Check the `output.log` file for detailed error messages and stack traces.
+* Issues related to agents and other systems: Check `_logs` directory for detailed error messages and stack traces.
 
 * Issues related to `Python quit unexpectedly`: Check [this stackoverflow article](https://stackoverflow.com/questions/59888499/macos-catalina-python-quit-unexpectedly-error).
 

{versionhq-1.2.1.1 → versionhq-1.2.1.3}/docs/core/Agent.md

@@ -1,8 +1,6 @@
 ---
 tags:
-  - HTML5
-  - JavaScript
-  - CSS
+  - Agent Network
 ---
 
 # Agent
@@ -14,7 +12,7 @@ Each agent has its unique knowledge and memory on the past task.
 You can create one and assign the task or reassign another task to the existing agent after fine-tuning.
 
 
-## Core usage
+## Quick Start
 
 By defining its role and goal in a simple sentence, the AI agent will be set up to run on <bold>`gpt-4o`</bold> by default.
 

versionhq-1.2.1.3/docs/core/task/evaluation.md

@@ -0,0 +1,8 @@
+---
+tags:
+  - Task Graph
+---
+
+# Evaluation
+
+Under drafting

{versionhq-1.2.1.1 → versionhq-1.2.1.3}/docs/core/task/response-field.md

@@ -1,23 +1,29 @@
 ---
 tags:
-  - HTML5
-  - JavaScript
-  - CSS
+  - Task Graph
 ---
 
-# ResponseField
+# Response Field
 
 <class>`class` versionhq.task.model.<bold>ResponseField<bold></class>
 
-A class to store response formats to create JSON response schema.
+A Pydantic class to store response formats to create JSON response schema.
 
 <hr/>
 
-## Core Usage
+## Variables
 
-`[var]`<bold>`title: str = None`</bold>
+| <div style="width:120px">**Variable**</div> | **Data Type** | **Default** | **Nullable** | **Description** |
+| :---               | :---  | :--- | :--- | :--- |
+| **`title`**   | str | None | False | Stores a field title. |
+| **`data_type`**       | Type | None | False | Stores data type of the response. |
+| **`items`** | Type | None | True | Stores data type of items in the list. Can leave it None when `data_type` is not list. |
+| **`properties`** | List[`ResponseField`] | None | True | Stores properties as a list of `ResponseFormat` objects when the `data_type` is dict. |
+| **`nullable`** |  bool | False | False | If the field is nullable. |
+| **`config`** | Dict[str, Any] | None | True | Stores other configs passed to response schema. |
 
-`[var]`<bold>`data_type: Type = None`</bold>
+
+**Quick Start**
 
 Define a response format with field titles and data types.
 

versionhq-1.2.1.3/docs/core/task/task-output.md

@@ -0,0 +1,79 @@
+---
+tags:
+  - Task Graph
+---
+
+# Task Output
+
+<class>`class` versionhq.task.model.<bold>TaskOutput<bold></class>
+
+A Pydantic class to store and manage response from the `Task` object.
+
+<hr />
+
+## Variables
+
+| <div style="width:120px">**Variable**</div> | **Data Type** | **Default** | **Nullable** | **Description** |
+| :---               | :---  | :--- | :--- | :--- |
+| **`task_id`**   | UUID  | uuid.uuid4() | False | Stores task `id` as an identifier. |
+| **`raw`**       | str   | None | False | Stores response in plane text format. `None` or `""` when the model returned errors.|
+| **`json_dict`** | Dict[str, Any] | None | False | Stores response in JSON serializable dictionary. When the system failed formatting or executing tasks without response_fields, `{ output: <res.raw> }` will be returned. |
+| **`pydantic`** | Type[`BaseModel`]  | None | True | Populates and stores Pydantic class defined in the `pydantic_output` field. `None` if `pydantic_output` is NOT given. |
+| **`tool_output`** |  Optional[Any] | None | True | Stores results from the tools of the task or agents ONLY when `tool_res_as_final` set as `True`. |
+| **`callback_output`** |  Optional[Any] | None | True | Stores results from callback functions if any. |
+| **`evaluation`** |  Optional[InstanceOf[`Evaluation`]] | None | True | Stores overall evaluations and usage of the task output. |
+
+
+The following snippet demonstrates the  `TaskOutput` object when the task is all-in with Pydantic response format, callbacks, tools, and evaluation.
+
+```python
+import versionhq as vhq
+from pydantic import BaseModel
+
+class CustomOutput(BaseModel):
+    test1: str
+    test2: list[str]
+
+def dummy_tool():
+    return "dummy"
+
+def summarize_response(message: str, test1: str, test2: list[str]) -> str:
+    return f"""{message}: {test1}, {", ".join(test2)}"""
+
+task = vhq.Task(
+    description="Research a topic to teach a kid aged 6 about math.",
+    pydantic_output=CustomOutput,
+    tools=[dummy_tool],
+    callback=summarize_response,
+    callback_kwargs=dict(message="Hi! Here is the result: "),
+    should_evaluate=True, # triggers evaluation
+    eval_criteria=["Uniquness", "Fit to audience",],
+
+)
+res = task.execute()
+
+assert res.task_id == task.id
+assert res.raw
+assert res.json_dict
+assert res.pydantic.test1 and res.pydantic.test2
+assert "Hi! Here is the result: " in res.callback_output
+assert res.pydantic.test1 in res.callback_output and ", ".join(res.pydantic.test2) in res.callback_output
+assert res.tool_output is None
+assert res.evaluation and isinstance(res.evaluation, vhq.Evaluation)
+```
+
+
+## Class Methods
+
+| <div style="width:120px">**Method**</div> | **Params** | **Returns** | **Description** |
+| :---               | :---  | :--- | :--- |
+| **`evaluate`**   | task: InstanceOf[`Task`]  | InstanceOf[`Evaluation`]  | Evaluates task output based on the criteria |
+
+Ref. <a href="/core/task/evaluation">Evaluation</a> class
+
+## Property
+
+| <div style="width:120px">**Property**</div> | **Returns** | **Description** |
+| :---               | :---  | :--- |
+| **`aggregate_score`**   | float | Calucurates weighted average eval scores of the task output. |
+| **`json_string`**       | str   | Returns `json_dict` in string format. |