divi 0.0.1b0__tar.gz → 0.0.1b1__tar.gz

{divi-0.0.1b0 → divi-0.0.1b1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: divi
-Version: 0.0.1b0
+Version: 0.0.1b1
 Summary: The Agent Platform for Observability & Evaluation
 License-File: LICENSE
 Requires-Python: >=3.11

divi-0.0.1b1/README.md

@@ -0,0 +1,72 @@
+<p align="center">
+  <a href="https://divine-agent.com/"><img width="128" height="128" src="https://raw.githubusercontent.com/Kaikaikaifang/divine-agent/main/docs/images/thinking-angel.png" alt='Divine Agent'></a>
+</p>
+
+<p align="center"><strong>Divine Agent</strong> <em>– Fully open-source agent observability. Simple. Clear.</em></p>
+
+<p align="center">
+<a href="https://pypi.org/project/divi/">
+    <img src="https://img.shields.io/pypi/v/divi.svg" alt="Package version">
+</a>
+</p>
+
+divine-agent is an observability tool for LLM-based agents, offering tracing, evaluation, and usage statistics.
+
+---
+
+> [!IMPORTANT]
+> **divine-agent is currently experimental** and may undergo significant changes at any time. This project is in active development, which means APIs and features might change without prior notice.
+>
+> We do not recommend using divine-agent in production environments until a stable release is available.
+
+## Install
+
+Requires Python 3.11+
+
+```shell
+pip install divi
+```
+
+## Usage
+
+1. Get API Key from [Web](https://www.divine-agent.com/dashboard/api-keys).
+2. Create a `.env` file and add the following line:
+  ```env
+  DIVI_API_KEY=your_api_key
+  ```
+3. Run the following code:
+  ```python
+  from divi import Score, obs_openai, observable
+  from dotenv import load_dotenv
+  from openai import OpenAI
+
+  load_dotenv()
+
+
+  class Pirate:
+      def __init__(self):
+          self.client = obs_openai(
+              OpenAI(),
+              name="Pirate",
+              scores=[Score.instruction_adherence],
+          )
+
+      @observable(name="Talk with pirate")
+      def talk(self, message: str):
+          """Talk like a pirate."""
+          res = self.client.chat.completions.create(
+              model="gpt-4o",
+              messages=[
+                  {"role": "developer", "content": "Talk like a pirate."},
+                  {
+                      "role": "user",
+                      "content": message,
+                  },
+              ],
+          )
+          return res.choices[0].message.content
+
+
+  pirate = Pirate()
+  pirate.talk("How do I check if a Python object is an instance of a class?")
+  ```

{divi-0.0.1b0 → divi-0.0.1b1}/apps/README.md

@@ -9,7 +9,3 @@
 | graphql | A GraphQL server | 4000 |
 | web | A Next.js web server | 4001 |
 | docs | A documentation server | 4002 |
-
-## Rules
-
-1. [Split project files by feature or route](https://nextjs.org/docs/app/getting-started/project-structure#split-project-files-by-feature-or-route)

divi-0.0.1b1/apps/docs/README.md

@@ -0,0 +1,3 @@
+# Divi Docs
+
+> This is a documentation site for the divine-agent project.

{divi-0.0.1b0 → divi-0.0.1b1}/apps/docs/docs.json

@@ -15,11 +15,7 @@
         "groups": [
           {
             "group": "Get Started",
-            "pages": ["en/introduction"]
-          },
-          {
-            "group": "Tutorials",
-            "pages": ["en/tutorials/self-refine"]
+            "pages": ["en/introduction", "en/quickstart/trace"]
           }
         ]
       },
@@ -28,11 +24,7 @@
         "groups": [
           {
             "group": "快速开始",
-            "pages": ["zh/introduction"]
-          },
-          {
-            "group": "使用案例",
-            "pages": ["zh/tutorials/self-refine"]
+            "pages": ["zh/introduction", "zh/quickstart/trace"]
           }
         ]
       }

{divi-0.0.1b0 → divi-0.0.1b1}/apps/docs/en/introduction.mdx

@@ -13,3 +13,5 @@ description: "Welcome to the docs of Divine Agent"
   src="/images/scholar-angel.png"
   alt="Hero Dark"
 />
+
+**Divine-Agent** is an observability tool for agents, offering tracing, evaluation, and usage statistics to help users observe the performance and behavior of their agents.

divi-0.0.1b1/apps/docs/en/quickstart/evaluation.mdx

@@ -0,0 +1,5 @@
+---
+title: 'Evaluation'
+description: 'Evaluate your agents in one line'
+---
+![Pirate](/images/pirate-example.png)

divi-0.0.1b1/apps/docs/en/quickstart/trace.mdx

@@ -0,0 +1,56 @@
+---
+title: 'Trace'
+description: 'Start observe your agents in under 5 minutes'
+---
+![Pirate](/images/pirate-example.png)
+
+## Install
+
+Requires Python 3.11+
+
+```shell
+pip install divi
+```
+
+## Steps
+
+1. Get API Key from [Web](https://www.divine-agent.com/dashboard/api-keys).
+2. Create a `.env` file and add the following line:
+  ```env
+  DIVI_API_KEY=your_api_key
+  ```
+3. Run the following code:
+  ```python Pirate Example {1,10-13,15} [expandable]
+  from divi import obs_openai, observable
+  from dotenv import load_dotenv
+  from openai import OpenAI
+
+  load_dotenv()
+
+
+  class Pirate:
+      def __init__(self):
+          self.client = obs_openai(
+              OpenAI(),
+              name="Pirate",
+          )
+
+      @observable(name="Talk with pirate")
+      def talk(self, message: str):
+          """Talk like a pirate."""
+          res = self.client.chat.completions.create(
+              model="gpt-4o",
+              messages=[
+                  {"role": "developer", "content": "Talk like a pirate."},
+                  {
+                      "role": "user",
+                      "content": message,
+                  },
+              ],
+          )
+          return res.choices[0].message.content
+
+
+  pirate = Pirate()
+  pirate.talk("How do I check if a Python object is an instance of a class?")
+  ```

{divi-0.0.1b0 → divi-0.0.1b1}/apps/docs/zh/development/roadmap.mdx

@@ -2,3 +2,5 @@
 title: '开发路线图'
 description: '开发路线图'
 ---
+
+- [ ] 完善相关文档

{divi-0.0.1b0 → divi-0.0.1b1}/apps/docs/zh/introduction.mdx

@@ -13,3 +13,5 @@ description: "欢迎使用 Divine Agent"
   src="/images/scholar-angel.png"
   alt="Hero Dark"
 />
+
+**神明代理人 (Divine Agent)** 是一个智能体可观测工具，提供追踪、评估和用量统计功能，帮助用户观察智能体的性能和行为。

divi-0.0.1b1/apps/docs/zh/quickstart/trace.mdx

@@ -0,0 +1,56 @@
+---
+title: '追踪'
+description: '5 分钟内开启智能体追踪'
+---
+![Pirate](/images/pirate-example.png)
+
+## Install
+
+需要 Python 版本 3.11+
+
+```shell
+pip install divi
+```
+
+## Steps
+
+1. 从 [Web](https://www.divine-agent.com/dashboard/api-keys) 获取 API Key.
+2. 创建 `.env` 文件并添加以下行:
+  ```env
+  DIVI_API_KEY=your_api_key
+  ```
+3. 运行以下代码:
+  ```python Pirate Example {1,10-13,15} [expandable]
+  from divi import obs_openai, observable
+  from dotenv import load_dotenv
+  from openai import OpenAI
+
+  load_dotenv()
+
+
+  class Pirate:
+      def __init__(self):
+          self.client = obs_openai(
+              OpenAI(),
+              name="Pirate",
+          )
+
+      @observable(name="Talk with pirate")
+      def talk(self, message: str):
+          """Talk like a pirate."""
+          res = self.client.chat.completions.create(
+              model="gpt-4o",
+              messages=[
+                  {"role": "developer", "content": "Talk like a pirate."},
+                  {
+                      "role": "user",
+                      "content": message,
+                  },
+              ],
+          )
+          return res.choices[0].message.content
+
+
+  pirate = Pirate()
+  pirate.talk("How do I check if a Python object is an instance of a class?")
+  ```

divi-0.0.1b1/apps/graphql/README.md

@@ -0,0 +1,3 @@
+# Divi GraphQL
+
+> This is the GraphQL API for the divine-agent project.

divi-0.0.1b1/apps/web/README.md

@@ -0,0 +1,3 @@
+# Divi Web
+
+> This is the web application for the divine-agent project.

divi-0.0.1b1/divi/README.md

@@ -0,0 +1,3 @@
+# DIVI SDK
+
+> this is the Python SDK for the divine-agent project.

{divi-0.0.1b0 → divi-0.0.1b1}/divi/__init__.py

@@ -1,10 +1,9 @@
 from typing import Optional
 
 from .decorators import obs_openai, observable
-from .evaluation import Evaluator, Score
+from .evaluation import Evaluator
 from .services import Auth, Core, DataPark
 from .session import Session
-from .signals import Kind
 
 name: str = "divi"
 
@@ -14,5 +13,5 @@ _auth: Optional[Auth] = None
 _datapark: Optional[DataPark] = None
 _evaluator: Optional[Evaluator] = None
 
-__version__ = "0.0.1b0"
-__all__ = ["obs_openai", "observable", "Score", "Kind"]
+__version__ = "0.0.1b1"
+__all__ = ["obs_openai", "observable"]

{divi-0.0.1b0 → divi-0.0.1b1}/divi/decorators/obs_openai.py

@@ -5,6 +5,7 @@ from typing import TYPE_CHECKING, TypeVar, Union
 from typing_extensions import Optional
 
 from divi.decorators.observable import observable
+from divi.evaluation.evaluator import EvaluatorConfig
 from divi.evaluation.scores import Score
 from divi.signals.span import Kind
 from divi.utils import is_async
@@ -19,10 +20,13 @@ def _get_observable_create(
     create: Callable,
     name: Optional[str] = None,
     scores: Optional[list[Score]] = None,
+    eval: Optional[EvaluatorConfig] = None,
 ) -> Callable:
     @functools.wraps(create)
     def observable_create(*args, stream: bool = False, **kwargs):
-        decorator = observable(kind=Kind.llm, name=name, scores=scores)
+        decorator = observable(
+            kind=Kind.llm, name=name, scores=scores, eval=eval
+        )
         return decorator(create)(*args, stream=stream, **kwargs)
 
     # TODO Async Observable Create
@@ -33,16 +37,13 @@ def obs_openai(
     client: C,
     name: Optional[str] = "Agent",
     scores: Optional[list[Score]] = None,
+    eval: Optional[EvaluatorConfig] = None,
 ) -> C:
     """Make OpenAI client observable."""
     client.chat.completions.create = _get_observable_create(
         client.chat.completions.create,
         name=name,
         scores=scores,
-    )
-    client.completions.create = _get_observable_create(
-        client.completions.create,
-        name=name,
-        scores=scores,
+        eval=eval,
     )
     return client

{divi-0.0.1b0 → divi-0.0.1b1}/divi/decorators/observable.py

@@ -15,6 +15,7 @@ from typing import (
 
 from divi.decorators.observe import observe
 from divi.evaluation.evaluate import evaluate_scores
+from divi.evaluation.evaluator import EvaluatorConfig
 from divi.evaluation.scores import Score
 from divi.session import SessionExtra
 from divi.signals.span import Kind, Span
@@ -43,6 +44,7 @@ def observable(
     *,
     name: Optional[str] = None,
     scores: Optional[list[Score]] = None,
+    eval: Optional[EvaluatorConfig] = None,
     metadata: Optional[Mapping[str, Any]] = None,
 ) -> Callable[[Callable[P, R]], WithSessionExtra[P, R]]: ...
 
@@ -56,6 +58,7 @@ def observable(
     name = kwargs.pop("name", None)
     metadata = kwargs.pop("metadata", None)
     scores: list[Score] = kwargs.pop("scores", None)
+    eval: EvaluatorConfig = kwargs.pop("eval", None)
 
     def decorator(func):
         @functools.wraps(func)
@@ -78,7 +81,9 @@ def observable(
 
             # 3. evaluate the scores if they are provided
             messages = kwargs.get("messages", [])
-            evaluate_scores(messages, outputs=result, scores=scores)
+            evaluate_scores(
+                messages, outputs=result, scores=scores, config=eval
+            )
 
             return result
 

divi-0.0.1b1/divi/evaluation/__init__.py

@@ -0,0 +1,4 @@
+from .evaluator import Evaluator, EvaluatorConfig
+from .scores import Score
+
+__all__ = ["Evaluator", "Score", "EvaluatorConfig"]

{divi-0.0.1b0 → divi-0.0.1b1}/divi/evaluation/evaluator.py

@@ -13,8 +13,9 @@ from divi.evaluation.scores import Score
 class EvaluatorConfig:
     def __init__(
         self,
-        model: str = "gpt-4.1-nano",
+        model: str = "gpt-4o",
         temperature: float = 0.5,
+        n_rounds: int = 5,
         max_concurrency: int = 10,
         api_key: Optional[str] = None,
         base_url: Optional[str] = None,
@@ -23,6 +24,7 @@ class EvaluatorConfig:
         self.api_key = api_key
         self.base_url = base_url
         self.temperature = temperature
+        self.n_rounds = n_rounds
         self.max_concurrency = max_concurrency
 
 
@@ -123,7 +125,11 @@ class Evaluator:
         return aggregated_results
 
     def evaluate_sync(
-        self, target: str, conversation: str, scores: list[Score], n_rounds: int
+        self,
+        target: str,
+        conversation: str,
+        scores: list[Score],
+        n_rounds: Optional[int] = None,
     ) -> List[EvaluationScore]:
         with concurrent.futures.ThreadPoolExecutor(
             max_workers=self.config.max_concurrency
@@ -132,7 +138,7 @@ class Evaluator:
                 executor.submit(
                     self._sync_evaluate_once, target, conversation, score
                 )
-                for _ in range(n_rounds)
+                for _ in range(n_rounds if n_rounds else self.config.n_rounds)
                 for score in scores
             ]
             evaluations = [
@@ -143,7 +149,11 @@ class Evaluator:
         )
 
     async def evaluate_async(
-        self, target: str, conversation: str, scores: list[Score], n_rounds: int
+        self,
+        target: str,
+        conversation: str,
+        scores: list[Score],
+        n_rounds: Optional[int] = None,
     ) -> List[EvaluationScore]:
         semaphore = asyncio.Semaphore(self.config.max_concurrency)
 
@@ -153,7 +163,11 @@ class Evaluator:
                     target, conversation, score
                 )
 
-        tasks = [sem_task(score) for _ in range(n_rounds) for score in scores]
+        tasks = [
+            sem_task(score)
+            for _ in range(n_rounds if n_rounds else self.config.n_rounds)
+            for score in scores
+        ]
         evaluations = await asyncio.gather(*tasks)
         return self._aggregate_results(
             [e for e in evaluations if e is not None]
@@ -164,7 +178,7 @@ class Evaluator:
         target: str,
         conversation: str,
         scores: list[Score],
-        n_rounds: int = 5,
+        n_rounds: Optional[int] = None,
         mode: Literal["sync", "async"] = "sync",
     ) -> List[EvaluationScore]:
         if mode == "async":

divi-0.0.1b1/packages/README.md

@@ -0,0 +1,15 @@
+# Packages
+
+> Js packages shared across the apps.
+
+## ui
+
+> Shared shadcn ui for the project.
+
+## typescript-config
+
+> Shared typescript configuration for the project.
+
+## graphql-client
+
+> Shared GraphQL client codes for the project.

divi-0.0.1b0/README.md

@@ -1,14 +0,0 @@
-# divine-agent
-
-**Agent Platform for Observability**
-
-## TODO
-
-- [ ] Web
-- [ ] Docs
-
-## Architecture
-
-![architecture](./docs/images/architecture.svg)
-
-![storage](./docs/images/storage.svg)

divi-0.0.1b0/apps/docs/README.md

@@ -1,32 +0,0 @@
-# Mintlify Starter Kit
-
-Click on `Use this template` to copy the Mintlify starter kit. The starter kit contains examples including
-
-- Guide pages
-- Navigation
-- Customizations
-- API Reference pages
-- Use of popular components
-
-### Development
-
-Install the [Mintlify CLI](https://www.npmjs.com/package/mintlify) to preview the documentation changes locally. To install, use the following command
-
-```
-npm i -g mintlify
-```
-
-Run the following command at the root of your documentation (where docs.json is)
-
-```
-mintlify dev
-```
-
-### Publishing Changes
-
-Install our Github App to auto propagate changes from your repo to your deployment. Changes will be deployed to production automatically after pushing to the default branch. Find the link to install on your dashboard. 
-
-#### Troubleshooting
-
-- Mintlify dev isn't running - Run `mintlify install` it'll re-install dependencies.
-- Page loads as a 404 - Make sure you are running in a folder with `docs.json`

divi-0.0.1b0/apps/docs/en/quickstart/trace.mdx

@@ -1,4 +0,0 @@
----
-title: 'Trace'
-description: 'Start observe your agents in under 5 minutes'
----

divi-0.0.1b0/apps/docs/zh/quickstart/trace.mdx

@@ -1,4 +0,0 @@
----
-title: '跟踪'
-description: 'Start observe your agents in under 5 minutes'
----

divi-0.0.1b0/apps/graphql/README.md

@@ -1,3 +0,0 @@
-# Catstronauts - server
-
-The starting point of the `server` code for Odyssey Lift-off I course.

divi-0.0.1b0/apps/web/README.md

@@ -1,36 +0,0 @@
-This is a [Next.js](https://nextjs.org) project bootstrapped with [`create-next-app`](https://nextjs.org/docs/app/api-reference/create-next-app).
-
-## Getting Started
-
-First, run the development server:
-
-```bash
-npm run dev
-# or
-yarn dev
-# or
-pnpm dev
-# or
-bun dev
-```
-
-Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
-
-You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.
-
-This project uses [`next/font`](https://nextjs.org/docs/app/building-your-application/optimizing/fonts) to automatically optimize and load Inter, a custom Google Font.
-
-## Learn More
-
-To learn more about Next.js, take a look at the following resources:
-
-- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
-- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
-
-You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js) - your feedback and contributions are welcome!
-
-## Deploy on Vercel
-
-The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.
-
-Check out our [Next.js deployment documentation](https://nextjs.org/docs/app/building-your-application/deploying) for more details.

divi-0.0.1b0/divi/evaluation/__init__.py

@@ -1,4 +0,0 @@
-from .evaluator import Evaluator
-from .scores import Score
-
-__all__ = ["Evaluator", "Score"]

divi-0.0.1b0/packages/README.md

@@ -1,15 +0,0 @@
-# Packages
-
-> Js packages shared across the apps.
-
-## eslint-config
-
-> Shared eslint configuration for the project.
-
-## typescript-config
-
-> Shared typescript configuration for the project.
-
-## [tailwind-config](https://github.com/vercel/turborepo/tree/main/examples/with-tailwind)
-
-> Shared tailwind configuration for the project.