vision-agent 0.2.30__py3-none-any.whl → 0.2.31__py3-none-any.whl

vision_agent-0.2.30.dist-info/METADATA

@@ -1,226 +0,0 @@
-Metadata-Version: 2.1
-Name: vision-agent
-Version: 0.2.30
-Summary: Toolset for Vision Agent
-Author: Landing AI
-Author-email: dev@landing.ai
-Requires-Python: >=3.9,<4.0
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: Python :: 3.9
-Classifier: Programming Language :: Python :: 3.10
-Classifier: Programming Language :: Python :: 3.11
-Requires-Dist: ipykernel (>=6.29.4,<7.0.0)
-Requires-Dist: langsmith (>=0.1.58,<0.2.0)
-Requires-Dist: moviepy (>=1.0.0,<2.0.0)
-Requires-Dist: nbclient (>=0.10.0,<0.11.0)
-Requires-Dist: nbformat (>=5.10.4,<6.0.0)
-Requires-Dist: numpy (>=1.21.0,<2.0.0)
-Requires-Dist: openai (>=1.0.0,<2.0.0)
-Requires-Dist: opencv-python-headless (>=4.0.0,<5.0.0)
-Requires-Dist: pandas (>=2.0.0,<3.0.0)
-Requires-Dist: pillow (>=10.0.0,<11.0.0)
-Requires-Dist: pydantic-settings (>=2.2.1,<3.0.0)
-Requires-Dist: requests (>=2.0.0,<3.0.0)
-Requires-Dist: rich (>=13.7.1,<14.0.0)
-Requires-Dist: scipy (>=1.13.0,<1.14.0)
-Requires-Dist: tabulate (>=0.9.0,<0.10.0)
-Requires-Dist: tqdm (>=4.64.0,<5.0.0)
-Requires-Dist: typing_extensions (>=4.0.0,<5.0.0)
-Project-URL: Homepage, https://landing.ai
-Project-URL: documentation, https://github.com/landing-ai/vision-agent
-Project-URL: repository, https://github.com/landing-ai/vision-agent
-Description-Content-Type: text/markdown
-
-<div align="center">
-    <img alt="vision_agent" height="200px" src="https://github.com/landing-ai/vision-agent/blob/main/assets/logo.jpg?raw=true">
-
-# 🔍🤖 Vision Agent
-
-[![](https://dcbadge.vercel.app/api/server/wPdN8RCYew?compact=true&style=flat)](https://discord.gg/wPdN8RCYew)
-![ci_status](https://github.com/landing-ai/vision-agent/actions/workflows/ci_cd.yml/badge.svg)
-[![PyPI version](https://badge.fury.io/py/vision-agent.svg)](https://badge.fury.io/py/vision-agent)
-![version](https://img.shields.io/pypi/pyversions/vision-agent)
-</div>
-
-Vision Agent is a library that helps you utilize agent frameworks for your vision tasks.
-Many current vision problems can easily take hours or days to solve, you need to find the
-right model, figure out how to use it, possibly write programming logic around it to
-accomplish the task you want or even more expensive, train your own model. Vision Agent
-aims to provide an in-seconds experience by allowing users to describe their problem in
-text and utilizing agent frameworks to solve the task for them. Check out our discord
-for updates and roadmaps!
-
-## Documentation
-
-- [Vision Agent Library Docs](https://landing-ai.github.io/vision-agent/)
-
-
-## Getting Started
-### Installation
-To get started, you can install the library using pip:
-
-```bash
-pip install vision-agent
-```
-
-Ensure you have an OpenAI API key and set it as an environment variable (if you are
-using Azure OpenAI please see the Azure setup section):
-
-```bash
-export OPENAI_API_KEY="your-api-key"
-```
-
-### Vision Agents
-You can interact with the agents as you would with any LLM or LMM model:
-
-```python
->>> from vision_agent.agent import VisionAgent
->>> agent = VisionAgent()
->>> agent("What percentage of the area of this jar is filled with coffee beans?", image="jar.jpg")
-"The percentage of area of the jar filled with coffee beans is 25%."
-```
-
-To better understand how the model came up with it's answer, you can also run it in
-debug mode by passing in the verbose argument:
-
-```python
->>> agent = VisionAgent(verbose=True)
-```
-
-You can also have it return the workflow it used to complete the task along with all
-the individual steps and tools to get the answer:
-
-```python
->>> resp, workflow = agent.chat_with_workflow([{"role": "user", "content": "What percentage of the area of this jar is filled with coffee beans?"}], image="jar.jpg")
->>> print(workflow)
-[{"task": "Segment the jar using 'grounding_sam_'.",
-  "tool": "grounding_sam_",
-  "parameters": {"prompt": "jar", "image": "jar.jpg"},
-  "call_results": [[
-    {
-      "labels": ["jar"],
-      "scores": [0.99],
-      "bboxes": [
-        [0.58, 0.2, 0.72, 0.45],
-      ],
-      "masks": "mask.png"
-    }
-  ]],
-  "answer": "The jar is located at [0.58, 0.2, 0.72, 0.45].",
-},
-{"visualize_output": "final_output.png"}]
-```
-
-You can also provide reference data for the model to utilize. For example, if you want
-to utilize VisualPromptCounting:
-
-```python
-agent(
-    "How many apples are in this image?",
-    image="apples.jpg",
-    reference_data={"bbox": [0.1, 0.11, 0.24, 0.25]},
-)
-```
-Where `[0.1, 0.11, 0.24, 0.25]` is the normalized bounding box coordinates of an apple.
-Similarly for DINOv you can provide a reference image and mask:
-
-```python
-agent(
-    "Can you detect all of the objects similar to the mask I've provided?",
-    image="image.jpg",
-    reference_data={"mask": "reference_mask.png", "image": "reference_image.png"},
-)
-```
-Here, `reference_mask.png` and `reference_image.png` in `reference_data` could be any
-image with it's corresponding mask that is the object you want to detect in `image.jpg`.
-You can find a demo app to generate masks for DINOv [here](examples/mask_app/).
-
-### Tools
-There are a variety of tools for the model or the user to use. Some are executed locally
-while others are hosted for you. You can also ask an LLM directly to build a tool for
-you. For example:
-
-```python
->>> import vision_agent as va
->>> llm = va.llm.OpenAILLM()
->>> detector = llm.generate_detector("Can you build a jar detector for me?")
->>> detector("jar.jpg")
-[{"labels": ["jar",],
-  "scores": [0.99],
-  "bboxes": [
-    [0.58, 0.2, 0.72, 0.45],
-  ]
-}]
-```
-
-#### Custom Tools
-You can also add your own custom tools for your vision agent to use:
-
-```python
-from vision_agent.tools import Tool, register_tool
-@register_tool
-class NumItems(Tool):
-   name = "num_items_"
-   description = "Returns the number of items in a list."
-   usage = {
-       "required_parameters": [{"name": "prompt", "type": "list"}],
-       "examples": [
-           {
-               "scenario": "How many items are in this list? ['a', 'b', 'c']",
-               "parameters": {"prompt": "['a', 'b', 'c']"},
-           }
-       ],
-   }
-   def __call__(self, prompt: list[str]) -> int:
-       return len(prompt)
-```
-This will register it with the list of tools Vision Agent has access to. It will be able
-to pick it based on the tool description and use it based on the usage provided. You can
-find an example that creates a custom tool for template matching [here](examples/custom_tools/).
-
-#### Tool List
-| Tool | Description |
-| --- | --- |
-| CLIP | CLIP is a tool that can classify or tag any image given a set of input classes or tags. |
-| ImageCaption| ImageCaption is a tool that can generate a caption for an image. |
-| GroundingDINO | GroundingDINO is a tool that can detect arbitrary objects with inputs such as category names or referring expressions. |
-| GroundingSAM | GroundingSAM is a tool that can detect and segment arbitrary objects with inputs such as category names or referring expressions. |
-| DINOv | DINOv is a tool that can detect arbitrary objects with using a referring mask. |
-| Crop | Crop crops an image given a bounding box and returns a file name of the cropped image. |
-| BboxArea | BboxArea returns the area of the bounding box in pixels normalized to 2 decimal places. |
-| SegArea | SegArea returns the area of the segmentation mask in pixels normalized to 2 decimal places. |
-| BboxIoU | BboxIoU returns the intersection over union of two bounding boxes normalized to 2 decimal places. |
-| SegIoU | SegIoU returns the intersection over union of two segmentation masks normalized to 2 decimal places. |
-| BoxDistance | BoxDistance returns the minimum distance between two bounding boxes normalized to 2 decimal places. |
-| MaskDistance | MaskDistance returns the minimum distance between two segmentation masks in pixel units |
-| BboxContains | BboxContains returns the intersection of two boxes over the target box area. It is good for check if one box is contained within another box. |
-| ExtractFrames | ExtractFrames extracts frames with motion from a video. |
-| ZeroShotCounting | ZeroShotCounting returns the total number of objects belonging to a single class in a given image. |
-| VisualPromptCounting | VisualPromptCounting returns the total number of objects belonging to a single class given an image and visual prompt. |
-| VisualQuestionAnswering | VisualQuestionAnswering is a tool that can explain the contents of an image and answer questions about the image. |
-| ImageQuestionAnswering | ImageQuestionAnswering is similar to VisualQuestionAnswering but does not rely on OpenAI and instead uses a dedicated model for the task. |
-| OCR | OCR returns the text detected in an image along with the location. |
-
-
-It also has a basic set of calculate tools such as add, subtract, multiply and divide.
-
-### Azure Setup
-If you want to use Azure OpenAI models, you can set the environment variable:
-
-```bash
-export AZURE_OPENAI_API_KEY="your-api-key"
-export AZURE_OPENAI_ENDPOINT="your-endpoint"
-```
-
-You can then run Vision Agent using the Azure OpenAI models:
-
-```python
->>> import vision_agent as va
->>> agent = va.agent.VisionAgent(
->>>     task_model=va.llm.AzureOpenAILLM(),
->>>     answer_model=va.lmm.AzureOpenAILMM(),
->>>     reflection_model=va.lmm.AzureOpenAILMM(),
->>> )
-```
-
-

vision_agent-0.2.30.dist-info/RECORD

@@ -1,36 +0,0 @@
-vision_agent/__init__.py,sha256=GVLHCeK_R-zgldpbcPmOzJat-BkadvkuRCMxDvTIcXs,108
-vision_agent/agent/__init__.py,sha256=jpmL6z5e4PFfQM21JbSsRwcERRXn58XFmURAMwWeoRM,249
-vision_agent/agent/agent.py,sha256=4buKL_7PA6q_Ktlf26FxfX0JxRGrL-swYk0xJuYNVz4,538
-vision_agent/agent/agent_coder.py,sha256=4Neo6qM9-J8sJ-PKqSaUHr28SYm43IjEvhDK8BfDosE,7006
-vision_agent/agent/agent_coder_prompts.py,sha256=CJe3v7xvHQ32u3RQAXQga_Tk_4UgU64RBAMHZ3S70KY,5538
-vision_agent/agent/easytool.py,sha256=oMHnBg7YBtIPgqQUNcZgq7uMgpPThs99_UnO7ERkMVg,11511
-vision_agent/agent/easytool_prompts.py,sha256=Bikw-PPLkm78dwywTlnv32Y1Tw6JMeC-R7oCnXWLcTk,4656
-vision_agent/agent/reflexion.py,sha256=4gz30BuFMeGxSsTzoDV4p91yE0R8LISXp28IaOI6wdM,10506
-vision_agent/agent/reflexion_prompts.py,sha256=G7UAeNz_g2qCb2yN6OaIC7bQVUkda4m3z42EG8wAyfE,9342
-vision_agent/agent/vision_agent.py,sha256=Rs7O0PXc2J9FlrpBa3UGs5NjqQT51Y507klQf9fC0UY,27281
-vision_agent/agent/vision_agent_prompts.py,sha256=MZSIwovYgB-f-kdJ6btaNDVXptJn47bfOL3-Zn6NiC0,8573
-vision_agent/agent/vision_agent_v2.py,sha256=t2D1mMUYEv1dFeMrkEUVbDEdArunb7F1ZeYB8qijU2w,15109
-vision_agent/agent/vision_agent_v2_prompts.py,sha256=b_0BMq6GrbGfl09MHrv4mj-mqyE1FxMl3Xq44qD4S1E,6161
-vision_agent/agent/vision_agent_v3.py,sha256=jPU__NueKQwFzIoJd0zzg6z9q7IDQa9QPaxt8Qlca98,12403
-vision_agent/agent/vision_agent_v3_prompts.py,sha256=ejedMNDluVYZjHOIXKN98LzX-pOHin2DJhCyZUWULNE,8070
-vision_agent/fonts/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-vision_agent/fonts/default_font_ch_en.ttf,sha256=1YM0Z3XqLDjSNbF7ihQFSAIUdjF9m1rtHiNC_6QosTE,1594400
-vision_agent/llm/__init__.py,sha256=BoUm_zSAKnLlE8s-gKTSQugXDqVZKPqYlWwlTLdhcz4,48
-vision_agent/llm/llm.py,sha256=_Klwngc35JdRuzezWe1P5BMBRkfRQSGJqNOtS44rM9s,5891
-vision_agent/lmm/__init__.py,sha256=nnNeKD1k7q_4vLb1x51O_EUTYaBgGfeiCx5F433gr3M,67
-vision_agent/lmm/lmm.py,sha256=gK90vMxh0OcGSuIZQikBkDXm4pfkdFk1R2y7rtWDl84,10539
-vision_agent/tools/__init__.py,sha256=dRHXGpjhItXZRQs0r_l3Z3bQIreaZaYP0CJrl8mOJxM,452
-vision_agent/tools/prompts.py,sha256=V1z4YJLXZuUl_iZ5rY0M5hHc_2tmMEUKr0WocXKGt4E,1430
-vision_agent/tools/tool_utils.py,sha256=wzRacbUpqk9hhfX_Y08rL8qP0XCN2w-8IZoYLi3Upn4,869
-vision_agent/tools/tools.py,sha256=pZc5dQlYINlV4nYbbzsDi3-wauA-fCeD2iGmJUMoUfE,47373
-vision_agent/tools/tools_v2.py,sha256=mio0A1l5QcyRC5IgaD4Trfqg7hFTZ8rOjx1dYivwb4Q,21585
-vision_agent/utils/__init__.py,sha256=xsHFyJSDbLdonB9Dh74cwZnVTiT__2OQF3Brd3Nmglc,116
-vision_agent/utils/execute.py,sha256=8_SfK-IkHH4lXF0JVyV7sDFszZn9HKsh1bFITKGCJ1g,3881
-vision_agent/utils/image_utils.py,sha256=_cdiS5YrLzqkq_ZgFUO897m5M4_SCIThwUy4lOklfB8,7700
-vision_agent/utils/sim.py,sha256=oUZ-6eu8Io-UNt9GXJ0XRKtP-Wc0sPWVzYGVpB2yDFk,3001
-vision_agent/utils/type_defs.py,sha256=BlI8ywWHAplC7kYWLvt4AOdnKpEW3qWEFm-GEOSkrFQ,1792
-vision_agent/utils/video.py,sha256=xTElFSFp1Jw4ulOMnk81Vxsh-9dTxcWUO6P9fzEi3AM,7653
-vision_agent-0.2.30.dist-info/LICENSE,sha256=xx0jnfkXJvxRnG63LTGOxlggYnIysveWIZ6H3PNdCrQ,11357
-vision_agent-0.2.30.dist-info/METADATA,sha256=uVj7XfG4Hat1Bed9FYM2dipIseooN4AHY-Tl4rSPOIg,9212
-vision_agent-0.2.30.dist-info/WHEEL,sha256=7Z8_27uaHI_UZAc4Uox4PpBhQ9Y5_modZXWMxtUi4NU,88
-vision_agent-0.2.30.dist-info/RECORD,,