divi 0.0.1b1__tar.gz → 0.0.1b3__tar.gz

divi-0.0.1b3/.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,38 @@
+---
+name: Bug report
+about: Create a report to help us improve
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Steps to reproduce the behavior:
+1. Go to '...'
+2. Click on '....'
+3. Scroll down to '....'
+4. See error
+
+**Expected behavior**
+A clear and concise description of what you expected to happen.
+
+**Screenshots**
+If applicable, add screenshots to help explain your problem.
+
+**Desktop (please complete the following information):**
+ - OS: [e.g. iOS]
+ - Browser [e.g. chrome, safari]
+ - Version [e.g. 22]
+
+**Smartphone (please complete the following information):**
+ - Device: [e.g. iPhone6]
+ - OS: [e.g. iOS8.1]
+ - Browser [e.g. stock browser, safari]
+ - Version [e.g. 22]
+
+**Additional context**
+Add any other context about the problem here.

divi-0.0.1b3/.github/ISSUE_TEMPLATE/feature_request.md

@@ -0,0 +1,20 @@
+---
+name: Feature request
+about: Suggest an idea for this project
+title: ''
+labels: ''
+assignees: ''
+
+---
+
+**Is your feature request related to a problem? Please describe.**
+A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
+
+**Describe the solution you'd like**
+A clear and concise description of what you want to happen.
+
+**Describe alternatives you've considered**
+A clear and concise description of any alternative solutions or features you've considered.
+
+**Additional context**
+Add any other context or screenshots about the feature request here.

divi-0.0.1b3/.github/pull_request_template.md

@@ -0,0 +1,12 @@
+<!-- Thanks for contributing to divine-agent! 🥳
+Given this is a project maintained by volunteers, please read this template to not waste your time, or ours! 😇 -->
+
+# Summary
+
+<!-- Write a small summary about what is happening here. -->
+
+# Checklist
+
+- [ ] I understand that this PR may be closed in case there was no previous discussion. (This doesn't apply to typos!)
+- [ ] I've added a test for each change that was introduced, and I tried as much as possible to make a single atomic change.
+- [ ] I've updated the documentation accordingly.

{divi-0.0.1b1 → divi-0.0.1b3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: divi
-Version: 0.0.1b1
+Version: 0.0.1b3
 Summary: The Agent Platform for Observability & Evaluation
 License-File: LICENSE
 Requires-Python: >=3.11

{divi-0.0.1b1 → divi-0.0.1b3}/README.md

@@ -10,6 +10,10 @@
 </a>
 </p>
 
+<p align="center">
+    <a href="./docs/README_ZH.md">中文</a> / English
+</p>
+
 divine-agent is an observability tool for LLM-based agents, offering tracing, evaluation, and usage statistics.
 
 ---
@@ -27,7 +31,7 @@ Requires Python 3.11+
 pip install divi
 ```
 
-## Usage
+## Trace
 
 1. Get API Key from [Web](https://www.divine-agent.com/dashboard/api-keys).
 2. Create a `.env` file and add the following line:
@@ -36,7 +40,7 @@ pip install divi
   ```
 3. Run the following code:
   ```python
-  from divi import Score, obs_openai, observable
+  from divi import obs_openai, observable
   from dotenv import load_dotenv
   from openai import OpenAI
 
@@ -48,7 +52,6 @@ pip install divi
           self.client = obs_openai(
               OpenAI(),
               name="Pirate",
-              scores=[Score.instruction_adherence],
           )
 
       @observable(name="Talk with pirate")

{divi-0.0.1b1 → divi-0.0.1b3}/apps/docs/docs.json

@@ -15,7 +15,11 @@
         "groups": [
           {
             "group": "Get Started",
-            "pages": ["en/introduction", "en/quickstart/trace"]
+            "pages": [
+              "en/introduction",
+              "en/quickstart/trace",
+              "en/quickstart/evaluation"
+            ]
           }
         ]
       },
@@ -24,7 +28,11 @@
         "groups": [
           {
             "group": "快速开始",
-            "pages": ["zh/introduction", "zh/quickstart/trace"]
+            "pages": [
+              "zh/introduction",
+              "zh/quickstart/trace",
+              "zh/quickstart/evaluation"
+            ]
           }
         ]
       }

divi-0.0.1b3/apps/docs/en/quickstart/evaluation.mdx

@@ -0,0 +1,34 @@
+---
+title: 'Evaluation'
+description: 'Evaluate your agents in one line'
+---
+import PirateEval from '/snippets/evaluation/pirate-eval.mdx';
+import PirateCustomEval from '/snippets/evaluation/pirate-custom-eval.mdx';
+
+<img className="block dark:hidden" src="/images/pirate-eval-example.png" />
+<img className="hidden dark:block" src="/images/pirate-eval-example-dark.png" />
+
+## Behavior Evaluation
+
+> Automate evaluation of instruction adherence and task completion for your [traced](./trace) Pirate agent.
+
+<PirateEval />
+
+## Custom Configuration
+
+| Property        | Type   | Default             | Description         |
+|-----------------|--------|---------------------|---------------------|
+| `model`         | str    | gpt-4o              | Model name used for evaluation |
+| `temperature`   | float  | 0.5                 | Temperature parameter for evaluation |
+| `n_rounds`      | int    | 5                   | Number of evaluation rounds |
+| `max_concurrency` | int  | 10                  | Maximum number of concurrent requests |
+| `api_key`       | str    | OPENAI_API_KEY      | API key for evaluation |
+| `base_url`      | str    | OPENAI_BASE_URL     | Service URL for evaluation |
+
+By default, `api_key` and `base_url` will use values from environment variables, and other options are configured as shown in the table above. You can customize the settings as follows:
+
+<PirateCustomEval />
+
+## Notes
+
+<Warning>The current evaluation functionality depends on OpenAI's [structured output](https://platform.openai.com/docs/guides/structured-outputs). Please ensure that your chosen [model](https://platform.openai.com/docs/models) supports this feature. We strongly recommend using `gpt-4o` or newer models to ensure evaluation effectiveness and compatibility.</Warning>

divi-0.0.1b3/apps/docs/en/quickstart/trace.mdx

@@ -0,0 +1,36 @@
+---
+title: 'Trace'
+description: 'Start observe your agents in under 5 minutes'
+---
+
+import PirateExample from '/snippets/pirate-example.mdx';
+
+<img className="block dark:hidden" src="/images/pirate-example.png" />
+<img className="hidden dark:block" src="/images/pirate-example-dark.png" />
+
+<Steps>
+    <Step title="Create an Account">
+        Create an account on [Divine Agent](https://www.divine-agent.com/signup?source=docs).
+    </Step>
+    <Step title="Install Divi">
+        Install the Divine Agent Python package:
+        ```shell
+        pip install divi
+        ```
+    </Step>
+    <Step title="Get API Key">
+      Get API Key from [Web](https://www.divine-agent.com/dashboard/api-keys).
+    </Step>
+    <Step title="Set up Environment">
+        Create a `.env` file and add the following line:
+        ```env
+        DIVI_API_KEY=your_api_key
+        OPENAI_BASE_URL=https://api.openai.com/v1
+        OPENAI_API_KEY=your_llm_api_key
+        ```
+    </Step>
+    <Step title="Trace Pirate Agent">
+        Run the following code:
+        <PirateExample />
+    </Step>
+</Steps>

divi-0.0.1b3/apps/docs/snippets/evaluation/pirate-custom-eval.mdx

@@ -0,0 +1,40 @@
+```python Pirate Example {2,15-18} [expandable]
+from divi import obs_openai, observable
+from divi.evaluation import EvaluatorConfig, Score
+from dotenv import load_dotenv
+from openai import OpenAI
+
+load_dotenv()
+
+
+class Pirate:
+    def __init__(self):
+        self.client = obs_openai(
+            OpenAI(),
+            name="Pirate",
+            scores=[Score.instruction_adherence, Score.task_completion],
+            eval=EvaluatorConfig(
+                model="gpt-4.1",
+                n_rounds=2,
+            ),
+        )
+
+    @observable(name="Talk with pirate")
+    def talk(self, message: str):
+        """Talk like a pirate."""
+        res = self.client.chat.completions.create(
+            model="gpt-4o",
+            messages=[
+                {"role": "developer", "content": "Talk like a pirate."},
+                {
+                    "role": "user",
+                    "content": message,
+                },
+            ],
+        )
+        return res.choices[0].message.content
+
+
+pirate = Pirate()
+pirate.talk("How do I check if a Python object is an instance of a class?")
+```

divi-0.0.1b3/apps/docs/snippets/evaluation/pirate-eval.mdx

@@ -0,0 +1,36 @@
+```python Pirate Example {2,14} [expandable]
+from divi import obs_openai, observable
+from divi.evaluation import Score
+from dotenv import load_dotenv
+from openai import OpenAI
+
+load_dotenv()
+
+
+class Pirate:
+    def __init__(self):
+        self.client = obs_openai(
+            OpenAI(),
+            name="Pirate",
+            scores=[Score.instruction_adherence, Score.task_completion],
+        )
+
+    @observable(name="Talk with pirate")
+    def talk(self, message: str):
+        """Talk like a pirate."""
+        res = self.client.chat.completions.create(
+            model="gpt-4o",
+            messages=[
+                {"role": "developer", "content": "Talk like a pirate."},
+                {
+                    "role": "user",
+                    "content": message,
+                },
+            ],
+        )
+        return res.choices[0].message.content
+
+
+pirate = Pirate()
+pirate.talk("How do I check if a Python object is an instance of a class?")
+```

divi-0.0.1b3/apps/docs/snippets/pirate-example.mdx

@@ -0,0 +1,34 @@
+```python Pirate Example {1,10-13,15} [expandable]
+from divi import obs_openai, observable
+from dotenv import load_dotenv
+from openai import OpenAI
+
+load_dotenv()
+
+
+class Pirate:
+    def __init__(self):
+        self.client = obs_openai(
+            OpenAI(),
+            name="Pirate",
+        )
+
+    @observable(name="Talk with pirate")
+    def talk(self, message: str):
+        """Talk like a pirate."""
+        res = self.client.chat.completions.create(
+            model="gpt-4o",
+            messages=[
+                {"role": "developer", "content": "Talk like a pirate."},
+                {
+                    "role": "user",
+                    "content": message,
+                },
+            ],
+        )
+        return res.choices[0].message.content
+
+
+pirate = Pirate()
+pirate.talk("How do I check if a Python object is an instance of a class?")
+```

divi-0.0.1b3/apps/docs/zh/quickstart/evaluation.mdx

@@ -0,0 +1,34 @@
+---
+title: '评估'
+description: '一行代码完成智能体评估'
+---
+import PirateEval from '/snippets/evaluation/pirate-eval.mdx';
+import PirateCustomEval from '/snippets/evaluation/pirate-custom-eval.mdx';
+
+<img className="block dark:hidden" src="/images/pirate-eval-example.png" />
+<img className="hidden dark:block" src="/images/pirate-eval-example-dark.png" />
+
+## 行为评估
+
+> 自动化评估[跟踪](./trace)的海盗 (Pirate) 智能体的指令遵从性和任务完成度。
+
+<PirateEval />
+
+## 自定义设置
+
+| Property        | Type   | Default             | Description         |
+|-----------------|--------|:-------------------:|---------------------|
+| `model`         | str    | gpt-4o              | 用于评估的模型名称     |
+| `temperature`   | float  | 0.5                 | 评估过程中的温度参数    |
+| `n_rounds`      | int    | 5                   | 评估的轮数            |
+| `max_concurrency` | int  | 10                  | 最大并发请求数         |
+| `api_key`       | str    | OPENAI_API_KEY      | 用于评估的 API 密钥    |
+| `base_url`      | str    | OPENAI_BASE_URL     | 评估所用的服务商 URL   |
+
+默认情况下，`api_key` 与 `base_url` 会使用环境变量中的值，其他选项配置如上表所示。您可以通过以下方式自定义设置：
+
+<PirateCustomEval />
+
+## 注意事项
+
+<Warning>当前评估功能依赖于 OpenAI 的[结构化输出](https://platform.openai.com/docs/guides/structured-outputs)，请确保您所选择的[模型](https://platform.openai.com/docs/models)支持该功能，我们强烈建议使用 `gpt-4o` 或更新版本的模型以确保评估效果和兼容性。</Warning>

divi-0.0.1b3/apps/docs/zh/quickstart/trace.mdx

@@ -0,0 +1,36 @@
+---
+title: '追踪'
+description: '5 分钟内开启智能体追踪'
+---
+
+import PirateExample from '/snippets/pirate-example.mdx';
+
+<img className="block dark:hidden" src="/images/pirate-example.png" />
+<img className="hidden dark:block" src="/images/pirate-example-dark.png" />
+
+<Steps>
+    <Step title="创建账户">
+        在 [Divine Agent](https://www.divine-agent.com/signup?source=docs) 上创建一个账户。
+    </Step>
+    <Step title="安装 Divi">
+        安装 Divine Agent Python 包:
+        ```shell
+        pip install divi
+        ```
+    </Step>
+    <Step title="获取 API Key">
+      从 [Web](https://www.divine-agent.com/dashboard/api-keys) 获取 API Key。
+    </Step>
+    <Step title="设置环境">
+        创建一个 `.env` 文件并添加以下行:
+        ```env
+        DIVI_API_KEY=your_api_key
+        OPENAI_BASE_URL=https://api.openai.com/v1
+        OPENAI_API_KEY=your_llm_api_key
+        ```
+    </Step>
+    <Step title="追踪 Pirate Agent">
+        运行以下代码:
+        <PirateExample />
+    </Step>
+</Steps>

{divi-0.0.1b1 → divi-0.0.1b3}/apps/web/app/dashboard/api-keys/components/create-dialog.tsx

@@ -35,7 +35,7 @@ export function CreateDialog() {
   return (
     <Dialog>
       <DialogTrigger asChild>
-        <Button size="sm">
+        <Button size="sm" className="h-7">
           <IconPlus />
           Create API Key
         </Button>

{divi-0.0.1b1 → divi-0.0.1b3}/apps/web/app/dashboard/layout.tsx

@@ -1,5 +1,3 @@
-import { AppSidebar } from './components/app-sidebar';
-import { SiteHeader } from './components/site-header';
 import { getCurrentUser } from '@/lib/server/auth';
 import { deleteSessionTokenCookie } from '@/lib/server/cookies';
 import {
@@ -9,6 +7,8 @@ import {
 import { cookies } from 'next/headers';
 import { redirect } from 'next/navigation';
 import type { CSSProperties, ReactNode } from 'react';
+import { AppSidebar } from './components/app-sidebar';
+import { SiteHeader } from './components/site-header';
 
 /**
  * Sign out
@@ -29,7 +29,8 @@ export default async function DashboardLayout({
   modal?: ReactNode;
 }) {
   const cookieStore = await cookies();
-  const defaultOpen = cookieStore.get('sidebar_state')?.value === 'true';
+  // defaultOpen is true if the cookie 'sidebar_state' is not set or is set to 'true'
+  const defaultOpen = cookieStore.get('sidebar_state')?.value !== 'false';
   const user = await getCurrentUser();
   if (!user) {
     return null;

{divi-0.0.1b1 → divi-0.0.1b3}/apps/web/components/Modal.tsx

@@ -18,6 +18,7 @@ import {
   DrawerTitle,
 } from '@workspace/ui/components/drawer';
 import { useMediaQuery } from '@workspace/ui/hooks/use-media-query';
+import { useIsMobile } from '@workspace/ui/hooks/use-mobile';
 import { Expand } from 'lucide-react';
 import { usePathname, useRouter } from 'next/navigation';
 import type * as React from 'react';
@@ -88,15 +89,20 @@ export function ResponsiveDrawer({
       setTimeout(router.back, 200);
     }
   };
+  const isMobile = useIsMobile();
+  if (isMobile === undefined) {
+    return null;
+  }
+  const _direction = isMobile ? 'bottom' : 'right';
 
   return (
     <Drawer
       defaultOpen={false}
       open={open}
       onOpenChange={handleOpenChange}
-      direction="right"
+      direction={_direction}
     >
-      <DrawerContent>
+      <DrawerContent className={isMobile ? 'h-[75vh]' : 'h-screen'}>
         <DrawerHeader>
           <DrawerTitle className="flex items-center justify-between">
             <Button variant="ghost" size="icon" asChild>

{divi-0.0.1b1 → divi-0.0.1b3}/divi/__init__.py

@@ -13,5 +13,5 @@ _auth: Optional[Auth] = None
 _datapark: Optional[DataPark] = None
 _evaluator: Optional[Evaluator] = None
 
-__version__ = "0.0.1b1"
+__version__ = "0.0.1b3"
 __all__ = ["obs_openai", "observable"]

{divi-0.0.1b1 → divi-0.0.1b3}/divi/evaluation/evaluator.py

@@ -19,6 +19,7 @@ class EvaluatorConfig:
         max_concurrency: int = 10,
         api_key: Optional[str] = None,
         base_url: Optional[str] = None,
+        language: str = "zh",
     ):
         self.model = model
         self.api_key = api_key
@@ -26,6 +27,7 @@ class EvaluatorConfig:
         self.temperature = temperature
         self.n_rounds = n_rounds
         self.max_concurrency = max_concurrency
+        self.language = language
 
 
 class EvaluationResult(BaseModel):
@@ -52,17 +54,22 @@ class Evaluator:
         )
 
     @staticmethod
-    def generate_prompt(target: str, conversation: str, score: Score) -> str:
+    def generate_prompt(
+        target: str, conversation: str, score: Score, language: str
+    ) -> str:
         return PROMPT_TEMPLATE.format(
             requirements=PRESET_PROMPT[score.value],
             target=target,
             conversation=conversation,
+            language=language,
         )
 
     def _sync_evaluate_once(
         self, target: str, conversation: str, score: Score
     ) -> Optional[EvaluationResult]:
-        prompt = self.generate_prompt(target, conversation, score)
+        prompt = self.generate_prompt(
+            target, conversation, score, self.config.language
+        )
         response = self.sync_client.beta.chat.completions.parse(
             model=self.config.model,
             messages=[{"role": "user", "content": prompt}],
@@ -77,7 +84,9 @@ class Evaluator:
     async def _async_evaluate_once(
         self, target: str, conversation: str, score: Score
     ) -> Optional[EvaluationResult]:
-        prompt = self.generate_prompt(target, conversation, score)
+        prompt = self.generate_prompt(
+            target, conversation, score, self.config.language
+        )
         response = await self.async_client.beta.chat.completions.parse(
             model=self.config.model,
             messages=[{"role": "user", "content": prompt}],

{divi-0.0.1b1 → divi-0.0.1b3}/divi/evaluation/prompts.py

@@ -8,7 +8,7 @@ PROMPT_TEMPLATE = (
     "Strictly output your answer in the following JSON format:\n"
     "{{\n"
     '  "judgment": bool,        # true if the response meets all requirements\n'
-    '  "reasoning": "string"    # concise explanation, hitting only the key points\n'
+    '  "reasoning": "string"    # concise explanation, in {language}, hitting only the key points\n'
     "}}\n"
     "Do not output anything else."
 )

{divi-0.0.1b1 → divi-0.0.1b3}/divi/signals/span.py

@@ -1,3 +1,4 @@
+import atexit
 import os
 import time
 from enum import Enum
@@ -62,6 +63,8 @@ class Span:
         """Start the span by recording the current time in nanoseconds."""
         self.start_time_unix_nano = time.time_ns()
         self.upsert_span()
+        # Register the end method to be called at exit
+        atexit.register(self.end)
 
     def end(self):
         """End the span by recording the end time in nanoseconds."""
@@ -69,6 +72,8 @@ class Span:
             raise ValueError("Span must be started before ending.")
         self.end_time_unix_nano = time.time_ns()
         self.upsert_span()
+        # Unregister the end method
+        atexit.unregister(self.end)
 
     def _add_node(self, trace_id: UUID4, parent_id: Optional[bytes] = None):
         """Add node for obs tree."""

{divi-0.0.1b1 → divi-0.0.1b3}/divi/signals/trace.py

@@ -1,3 +1,4 @@
+import atexit
 from datetime import UTC, datetime
 from typing import Optional
 from uuid import uuid4
@@ -63,6 +64,8 @@ class Trace:
         """Start the trace by recording the current time in nanoseconds."""
         self.start_time = datetime.now(UTC).isoformat()
         self.upsert_trace()
+        # Register the end method to be called on exit
+        atexit.register(self.end)
 
     def end(self):
         """End the trace by recording the end time in nanoseconds."""
@@ -70,6 +73,8 @@ class Trace:
             raise ValueError("Span must be started before ending.")
         self.end_time = datetime.now(UTC).isoformat()
         self.upsert_trace()
+        # Unregister the end method to prevent multiple calls
+        atexit.unregister(self.end)
 
     def upsert_trace(self):
         """Upsert trace with datapark."""