vision-agent 0.2.175__tar.gz → 0.2.177__tar.gz

{vision_agent-0.2.175 → vision_agent-0.2.177}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: vision-agent
-Version: 0.2.175
+Version: 0.2.177
 Summary: Toolset for Vision Agent
 Author: Landing AI
 Author-email: dev@landing.ai
@@ -53,25 +53,25 @@ Description-Content-Type: text/markdown
 ![version](https://img.shields.io/pypi/pyversions/vision-agent)
 </div>
 
-Vision Agent is a library that helps you utilize agent frameworks to generate code to
+VisionAgent is a library that helps you utilize agent frameworks to generate code to
 solve your vision task. Many current vision problems can easily take hours or days to
 solve, you need to find the right model, figure out how to use it and program it to
-accomplish the task you want. Vision Agent aims to provide an in-seconds experience by
+accomplish the task you want. VisionAgent aims to provide an in-seconds experience by
 allowing users to describe their problem in text and have the agent framework generate
 code to solve the task for them. Check out our discord for updates and roadmaps!
 
 ## Table of Contents
 - [🚀Quick Start](#quick-start)
 - [📚Documentation](#documentation)
-- [🔍🤖Vision Agent](#vision-agent-basic-usage)
+- [🔍🤖VisionAgent](#vision-agent-basic-usage)
 - [🛠️Tools](#tools)
 - [🤖LMMs](#lmms)
-- [💻🤖Vision Agent Coder](#vision-agent-coder)
+- [💻🤖VisionAgent Coder](#vision-agent-coder)
 - [🏗️Additional Backends](#additional-backends)
 
 ## Quick Start
 ### Web Application
-The fastest way to test out Vision Agent is to use our web application. You can find it
+The fastest way to test out VisionAgent is to use our web application. You can find it
 [here](https://va.landing.ai/).
 
 
@@ -108,9 +108,9 @@ be images or video files.
 
 ## Documentation
 
-[Vision Agent Library Docs](https://landing-ai.github.io/vision-agent/)
+[VisionAgent Library Docs](https://landing-ai.github.io/vision-agent/)
 
-## Vision Agent Basic Usage
+## VisionAgent Basic Usage
 ### Chatting and Message Formats
 `VisionAgent` is an agent that can chat with you and call other tools or agents to
 write vision code for you. You can interact with it like you would ChatGPT or any other
@@ -292,7 +292,7 @@ response = lmm(
 )
 ```
 
-## Vision Agent Coder
+## VisionAgent Coder
 Underneath the hood, `VisionAgent` uses `VisionAgentCoder` to generate code to solve
 vision tasks. You can use `VisionAgentCoder` directly to generate code if you want:
 
@@ -488,7 +488,7 @@ export AZURE_OPENAI_EMBEDDING_MODEL_DEPLOYMENT_NAME="your_embedding_model_deploy
 
 > NOTE: make sure your Azure model deployment have enough quota (token per minute) to support it. The default value 8000TPM is not enough.
 
-You can then run Vision Agent using the Azure OpenAI models:
+You can then run VisionAgent using the Azure OpenAI models:
 
 ```python
 import vision_agent as va

{vision_agent-0.2.175 → vision_agent-0.2.177}/README.md

@@ -11,25 +11,25 @@
 ![version](https://img.shields.io/pypi/pyversions/vision-agent)
 </div>
 
-Vision Agent is a library that helps you utilize agent frameworks to generate code to
+VisionAgent is a library that helps you utilize agent frameworks to generate code to
 solve your vision task. Many current vision problems can easily take hours or days to
 solve, you need to find the right model, figure out how to use it and program it to
-accomplish the task you want. Vision Agent aims to provide an in-seconds experience by
+accomplish the task you want. VisionAgent aims to provide an in-seconds experience by
 allowing users to describe their problem in text and have the agent framework generate
 code to solve the task for them. Check out our discord for updates and roadmaps!
 
 ## Table of Contents
 - [🚀Quick Start](#quick-start)
 - [📚Documentation](#documentation)
-- [🔍🤖Vision Agent](#vision-agent-basic-usage)
+- [🔍🤖VisionAgent](#vision-agent-basic-usage)
 - [🛠️Tools](#tools)
 - [🤖LMMs](#lmms)
-- [💻🤖Vision Agent Coder](#vision-agent-coder)
+- [💻🤖VisionAgent Coder](#vision-agent-coder)
 - [🏗️Additional Backends](#additional-backends)
 
 ## Quick Start
 ### Web Application
-The fastest way to test out Vision Agent is to use our web application. You can find it
+The fastest way to test out VisionAgent is to use our web application. You can find it
 [here](https://va.landing.ai/).
 
 
@@ -66,9 +66,9 @@ be images or video files.
 
 ## Documentation
 
-[Vision Agent Library Docs](https://landing-ai.github.io/vision-agent/)
+[VisionAgent Library Docs](https://landing-ai.github.io/vision-agent/)
 
-## Vision Agent Basic Usage
+## VisionAgent Basic Usage
 ### Chatting and Message Formats
 `VisionAgent` is an agent that can chat with you and call other tools or agents to
 write vision code for you. You can interact with it like you would ChatGPT or any other
@@ -250,7 +250,7 @@ response = lmm(
 )
 ```
 
-## Vision Agent Coder
+## VisionAgent Coder
 Underneath the hood, `VisionAgent` uses `VisionAgentCoder` to generate code to solve
 vision tasks. You can use `VisionAgentCoder` directly to generate code if you want:
 
@@ -446,7 +446,7 @@ export AZURE_OPENAI_EMBEDDING_MODEL_DEPLOYMENT_NAME="your_embedding_model_deploy
 
 > NOTE: make sure your Azure model deployment have enough quota (token per minute) to support it. The default value 8000TPM is not enough.
 
-You can then run Vision Agent using the Azure OpenAI models:
+You can then run VisionAgent using the Azure OpenAI models:
 
 ```python
 import vision_agent as va

{vision_agent-0.2.175 → vision_agent-0.2.177}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "poetry.core.masonry.api"
 
 [tool.poetry]
 name = "vision-agent"
-version = "0.2.175"
+version = "0.2.177"
 description = "Toolset for Vision Agent"
 authors = ["Landing AI <dev@landing.ai>"]
 readme = "README.md"

{vision_agent-0.2.175 → vision_agent-0.2.177}/vision_agent/agent/vision_agent.py

@@ -212,6 +212,19 @@ def add_step_descriptions(response: Dict[str, Any]) -> Dict[str, Any]:
     return response
 
 
+def setup_artifacts() -> Artifacts:
+    # this is setting remote artifacts path
+    sandbox = os.environ.get("CODE_SANDBOX_RUNTIME", None)
+    if sandbox is None or sandbox == "local":
+        remote = WORKSPACE / "artifacts.pkl"
+    elif sandbox == "e2b":
+        remote = Path("/home/user/artifacts.pkl")
+    else:
+        raise ValueError(f"Unknown code sandbox runtime {sandbox}")
+    artifacts = Artifacts(remote, Path(os.getcwd()) / "artifacts.pkl")
+    return artifacts
+
+
 def new_format_to_old_format(new_format: Dict[str, Any]) -> Dict[str, Any]:
     thoughts = new_format["thinking"] if new_format["thinking"] is not None else ""
     response = new_format["response"] if new_format["response"] is not None else ""
@@ -384,8 +397,7 @@ class VisionAgent(Agent):
             raise ValueError("chat cannot be empty")
 
         if not artifacts:
-            # this is setting remote artifacts path
-            artifacts = Artifacts("", "")
+            artifacts = setup_artifacts()
 
         # NOTE: each chat should have a dedicated code interpreter instance to avoid concurrency issues
         code_interpreter = (
@@ -400,7 +412,7 @@ class VisionAgent(Agent):
 
         if code_interpreter.remote_path != artifacts.remote_save_path.parent:
             raise ValueError(
-                f"Code interpreter remote path {code_interpreter.remote_path} does not match {artifacts.remote_save_path.parent}"
+                f"Code interpreter remote path {code_interpreter.remote_path} does not match artifacts remote path {artifacts.remote_save_path.parent}"
             )
 
         with code_interpreter:
@@ -415,7 +427,7 @@ class VisionAgent(Agent):
                         artifacts.artifacts[Path(media).name] = open(media, "rb").read()
 
                         media_remote_path = (
-                            Path(code_interpreter.remote_path) / Path(media).name
+                            Path(artifacts.remote_save_path.parent) / Path(media).name
                         )
                         chat_i["content"] += f" Media name {media_remote_path}"  # type: ignore
                         media_list.append(media_remote_path)

{vision_agent-0.2.175 → vision_agent-0.2.177}/vision_agent/tools/__init__.py

@@ -1,5 +1,17 @@
 from typing import Callable, List, Optional
 
+from .meta_tools import (
+    create_code_artifact,
+    edit_code_artifact,
+    edit_vision_code,
+    generate_vision_code,
+    get_tool_descriptions,
+    list_artifacts,
+    object_detection_fine_tuning,
+    open_code_artifact,
+    use_object_detection_fine_tuning,
+    view_media_artifact,
+)
 from .prompts import CHOOSE_PARAMS, SYSTEM_PROMPT
 from .tool_utils import get_tool_descriptions_by_names
 from .tools import (
@@ -65,7 +77,11 @@ def register_tool(imports: Optional[List] = None) -> Callable:
     def decorator(tool: Callable) -> Callable:
         import inspect
 
-        from .tools import get_tool_descriptions, get_tools_df, get_tools_info
+        from .tools import (  # noqa: F811
+            get_tool_descriptions,
+            get_tools_df,
+            get_tools_info,
+        )
 
         global TOOLS, TOOLS_DF, TOOL_DESCRIPTIONS, TOOL_DOCSTRING, TOOLS_INFO
 

{vision_agent-0.2.175 → vision_agent-0.2.177}/vision_agent/tools/meta_tools.py

@@ -76,6 +76,14 @@ class Artifacts:
     def __init__(
         self, remote_save_path: Union[str, Path], local_save_path: Union[str, Path]
     ) -> None:
+        """Initializes the Artifacts object with it's remote and local save paths.
+
+        Parameters:
+            remote_save_path (Union[str, Path]): The path to save the artifacts in the
+                remote environment. For example "/home/user/artifacts.pkl".
+            local_save_path (Union[str, Path]): The path to save the artifacts in the
+                local environment. For example "/Users/my_user/workspace/artifacts.pkl".
+        """
         self.remote_save_path = Path(remote_save_path)
         self.local_save_path = Path(local_save_path)
         self.artifacts: Dict[str, Any] = {}
@@ -85,31 +93,46 @@ class Artifacts:
     def load(
         self,
         artifacts_path: Union[str, Path],
-        load_to: Optional[Union[str, Path]] = None,
+        load_to_dir: Optional[Union[str, Path]] = None,
     ) -> None:
-        """Loads are artifacts into the load_to path. If load_to is None, it will load
-        into remote_save_path. If an artifact value is None it will skip loading it.
+        """Loads are artifacts into the load_to_dir directory. If load_to_dir is None,
+        it will load into remote_save_path directory. If an artifact value is None it
+        will skip loading it.
 
         Parameters:
-            artifacts_path (Union[str, Path]): The file path to load the artifacts from
+            artifacts_path (Union[str, Path]): The file path to load the artifacts from.
+                If you are in the remote environment this would be remote_save_path, if
+                you are in the local environment this would be local_save_path.
+            load_to_dir (Optional[Union[str, Path]): The directory to load the artifacts
+                into. If None, it will load into remote_save_path directory.
         """
         with open(artifacts_path, "rb") as f:
             self.artifacts = pkl.load(f)
 
-        load_to = self.remote_save_path.parent if load_to is None else Path(load_to)
+        load_to_dir = (
+            self.remote_save_path.parent if load_to_dir is None else Path(load_to_dir)
+        )
 
         for k, v in self.artifacts.items():
             if v is not None:
                 mode = "w" if isinstance(v, str) else "wb"
-                with open(load_to / k, mode) as f:
+                with open(load_to_dir / k, mode) as f:
                     f.write(v)
 
-    def show(self, uploaded_file_path: Optional[Union[str, Path]] = None) -> str:
-        """Shows the artifacts that have been loaded and their remote save paths."""
+    def show(self, uploaded_file_dir: Optional[Union[str, Path]] = None) -> str:
+        """Prints out the artifacts and the directory they have been loaded to. If you
+        pass in upload_file_dir, it will show the artifacts have been loaded to the
+        upload_file_dir directory. If you don't pass in upload_file_dir, it will show
+        the artifacts have been loaded to the remote_save_path directory.
+
+        Parameters:
+            uploaded_file_dir (Optional[Union[str, Path]): The directory the artifacts
+                have been loaded to.
+        """
         loaded_path = (
-            Path(uploaded_file_path)
-            if uploaded_file_path is not None
-            else self.remote_save_path
+            Path(uploaded_file_dir)
+            if uploaded_file_dir is not None
+            else self.remote_save_path.parent
         )
         output_str = "[Artifacts loaded]\n"
         for k in self.artifacts.keys():
@@ -121,6 +144,9 @@ class Artifacts:
         return output_str
 
     def save(self, local_path: Optional[Union[str, Path]] = None) -> None:
+        """Saves the artifacts to the local_save_path directory. If local_path is None,
+        it will save to the local_save_path directory.
+        """
         save_path = Path(local_path) if local_path is not None else self.local_save_path
         with open(save_path, "wb") as f:
             pkl.dump(self.artifacts, f)