drtrace 0.4.0 → 0.5.0

package/agents/daemon-method-selection.md

@@ -1,83 +1,97 @@
 # Daemon Method Selection Guide
-
 **Purpose**: Guide for AI agents to choose the right method for daemon interaction.
+**Key Principle**: For AI agents, prefer interaction with daemon in this order (most preferred to least, if a method is unavailable, fall back to the next):
+- **CLI**: if available on client machine (check `## CLI Approach` below).
+- **HTTP/curl**: for complex interactions without dependencies (check `## HTTP/curl Approach` below).
+- **Python code**: when rich queries are needed (check `## Python code Approach` below).
+## CLI Approach
+If `drtrace` CLI is installed on the client machine:
+- Check with: `drtrace --version`
+- Prefer CLI commands: `drtrace grep`, `drtrace tail`, `drtrace status`
+- Benefit: no HTTP overhead and faster queries
+### Query Filter Rule: Mutually Exclusive
+When querying logs, use **either** `message_contains` **or** `message_regex`, **NOT both**.
+
+| Filter             | Use Case         | Example                                       |
+| ------------------ | ---------------- | --------------------------------------------- |
+| `message_contains` | Substring search | "timeout" matches "Connection timeout error"  |
+| `message_regex`    | Regex patterns   | "(db\|cache).*timeout" matches service errors |
+**Error if both used**: API returns 400 "Cannot use both filters".
+### Examples
+**CLI - Substring Search**:
+```bash
+drtrace grep "timeout" --since 1h
+```
+**CLI - Regex Search** (with `-E`):
+```bash
+drtrace grep -E "(db|cache).*timeout" --since 1h
+```
+## HTTP/curl Approach
+### Check Daemon Status
 
-**Key Principle**: For AI agents, prefer **HTTP/curl first** (simpler, no dependencies), then Python, then CLI.
-
----
-
-## Priority Order for AI Agents
-
-| Priority | Method | When to Use |
-|----------|--------|-------------|
-| **1st** | HTTP/curl | Default - works without dependencies |
-| **2nd** | Python code | When rich SDK features needed |
-| **3rd** | CLI commands | Last resort - requires subprocess |
-
-### Why HTTP-First for AI Agents?
-
-- **No dependencies**: Just HTTP requests - works in any environment
-- **Simpler**: curl commands are self-contained and portable
-- **Faster**: No import overhead or Python interpreter startup
-- **Universal**: Works from any language or tool
-
----
-
-## Discovery First: OpenAPI Schema
-
+```bash
+curl http://localhost:8001/status
+```
+### Discovery API via OpenAPI Schema
 **CRITICAL**: Before making any API call, fetch `/openapi.json` to discover:
 - Available endpoints
 - Correct field names (e.g., `ts` not `timestamp`)
 - Request/response schemas
-
 ```bash
 # Always fetch schema first
 curl http://localhost:8001/openapi.json
 ```
-
-```python
-import requests
-
-base_url = "http://localhost:8001"
-
-# Fetch schema to discover endpoints and field names
-schema = requests.get(f"{base_url}/openapi.json", timeout=5).json()
-paths = schema.get("paths", {})
-components = schema.get("components", {}).get("schemas", {})
-
-# Use discovered field names, not hardcoded ones
+### Common Endpoints (for Quick Reference)
+**Note**: Always verify via `/openapi.json` - this list is for reference only (*API field names can change between versions. Using OpenAPI ensures compatibility*).
+
+| Endpoint               | Method | Purpose              |
+| ---------------------- | ------ | -------------------- |
+| `/status`              | GET    | Health check         |
+| `/openapi.json`        | GET    | API schema           |
+| `/logs/query`          | GET    | Query logs           |
+| `/logs/ingest`         | POST   | Ingest logs          |
+| `/analysis/why`        | GET    | Root cause analysis  |
+| `/analysis/time-range` | GET    | Time range analysis  |
+| `/help/guide/start`    | POST   | Start setup guide    |
+| `/help/guide/current`  | GET    | Current setup step   |
+| `/help/troubleshoot`   | POST   | Troubleshooting help |
+### Examples
+**HTTP - Substring Search**:
+```bash
+curl "http://localhost:8001/logs/query?message_contains=timeout&since=1h"
 ```
-
-**Why this matters**: API field names can change between versions. Using OpenAPI ensures compatibility.
-
----
-
-## Method 1: HTTP/curl (Preferred)
-
-### Check Daemon Status
-
+**HTTP - Regex Search**:
 ```bash
-curl http://localhost:8001/status
+curl "http://localhost:8001/logs/query?message_regex=(db|cache).*timeout&since=1h"
 ```
-
-### Query Logs
-
+**HTTP - Query Logs**:
 ```bash
 # Get logs from last 5 minutes
 START_TS=$(python3 -c "import time; print(time.time() - 300)")
 END_TS=$(python3 -c "import time; print(time.time())")
-
 curl "http://localhost:8001/logs/query?start_ts=${START_TS}&end_ts=${END_TS}&application_id=myapp&limit=100"
 ```
-
-### Root Cause Analysis
-
+**HTTP - Query with filter**:
 ```bash
 curl "http://localhost:8001/analysis/why?application_id=myapp&start_ts=${START_TS}&end_ts=${END_TS}&min_level=ERROR"
 ```
+## Python code Approach
+Use when you need rich SDK features or the HTTP method fails.
+### Examples
+**Get OpenAPI json for API information**
+```python
+import requests
 
-### Python requests (HTTP method)
+base_url = "http://localhost:8001"
 
+# Fetch schema to discover endpoints and field names
+schema = requests.get(f"{base_url}/openapi.json", timeout=5).json()
+paths = schema.get("paths", {})
+components = schema.get("components", {}).get("schemas", {})
+
+# Use discovered field names, not hardcoded ones
+```
+**Python requests (HTTP method)**
 ```python
 import requests
 import time
@@ -109,202 +123,31 @@ for log in logs:
     level = log.get("level")
     message = log.get("message")
 ```
-
----
-
-## Method 2: Python Code (Fallback)
-
-Use when you need rich SDK features or the HTTP method fails.
-
-### Log Analysis
-
-```python
-from drtrace_service.agent_interface import process_agent_query, check_daemon_status
-import asyncio
-
-# Check daemon first
-status = await check_daemon_status()
-if not status.get("available"):
-    print("Daemon not available")
-
-# Process query - returns formatted markdown
-response = await process_agent_query("explain error from 9:00 to 10:00 for app myapp")
-print(response)
-
-# Non-async context
-response = asyncio.run(process_agent_query("show logs from last 5 minutes"))
-```
-
-### Setup Help
-
-```python
-from drtrace_service.help_agent_interface import (
-    start_setup_guide,
-    get_current_step,
-    complete_step,
-    troubleshoot,
-)
-from pathlib import Path
-import asyncio
-
-project_root = Path(".")
-
-# Start setup guide
-guide = await start_setup_guide(language="python", project_root=project_root)
-
-# Get current step
-current = await get_current_step(project_root=project_root)
-
-# Mark step complete
-next_step = await complete_step(step_number=1, project_root=project_root)
-
-# Troubleshoot
-help_text = await troubleshoot("daemon not connecting", project_root=project_root)
-```
-
-### Setup Suggestions
-
-```python
-from drtrace_service.setup_agent_interface import analyze_and_suggest, suggest_for_language
-from pathlib import Path
-import asyncio
-
-project_root = Path(".")
-
-# Get setup suggestions
-suggestions = await analyze_and_suggest(project_root)
-
-# Language-specific suggestions
-python_suggestions = await suggest_for_language("python", project_root)
-cpp_suggestions = await suggest_for_language("cpp", project_root)
-```
-
----
-
-## Method 3: CLI Commands (Last Resort)
-
-Use only when HTTP and Python methods are unavailable.
-
-### Check Status
-
-```bash
-python -m drtrace_service status
-```
-
-### Analyze Errors
-
-```bash
-python -m drtrace_service why --application-id myapp --since 5m --min-level ERROR
-```
-
-### Setup Help
-
-```bash
-python -m drtrace_service help guide start --language python --project-root /path/to/project
-python -m drtrace_service help guide current --project-root /path/to/project
-```
-
----
-
-## Error Handling and Fallback Chain
-
-```python
-import requests
-
-def interact_with_daemon(query: str) -> str:
-    """Try methods in order: HTTP -> Python -> CLI"""
-    base_url = "http://localhost:8001"
-
-    # Method 1: HTTP (Preferred)
-    try:
-        # Check status first
-        status = requests.get(f"{base_url}/status", timeout=2)
-        if status.status_code == 200:
-            # Make actual request...
-            return "Success via HTTP"
-    except requests.exceptions.RequestException:
-        pass  # Fall through to Python
-
-    # Method 2: Python (Fallback)
-    try:
-        from drtrace_service.agent_interface import process_agent_query
-        import asyncio
-        response = asyncio.run(process_agent_query(query))
-        return response
-    except ImportError:
-        pass  # Fall through to CLI
-
-    # Method 3: CLI (Last Resort)
-    import subprocess
-    result = subprocess.run(
-        ["python", "-m", "drtrace_service", "status"],
-        capture_output=True,
-        text=True
-    )
-    if result.returncode == 0:
-        return result.stdout
-
-    # All methods failed
-    return """
-    Daemon is not available.
-
-    Next Steps:
-    1. Start daemon: python -m drtrace_service
-    2. Verify: curl http://localhost:8001/status
-    """
-```
-
----
-
 ## When to Use Each Method
 
-| Scenario | Recommended Method |
-|----------|-------------------|
-| Simple queries (status, logs) | HTTP/curl |
-| Complex analysis | HTTP or Python |
-| Rich markdown output needed | Python SDK |
-| Running outside Python | HTTP/curl |
-| Setup guide interaction | Python SDK |
-| Debugging integration | CLI commands |
-| Script automation | HTTP/curl |
-
----
-
-## Quick Reference: Common Endpoints
-
-**Note**: Always verify via `/openapi.json` - this list is for reference only.
-
-| Endpoint | Method | Purpose |
-|----------|--------|---------|
-| `/status` | GET | Health check |
-| `/openapi.json` | GET | API schema |
-| `/logs/query` | GET | Query logs |
-| `/logs/ingest` | POST | Ingest logs |
-| `/analysis/why` | GET | Root cause analysis |
-| `/analysis/time-range` | GET | Time range analysis |
-| `/help/guide/start` | POST | Start setup guide |
-| `/help/guide/current` | GET | Current setup step |
-| `/help/troubleshoot` | POST | Troubleshooting help |
-
----
-
-## Important: Timestamps Are UTC
-
+| Scenario                      | Recommended Method |
+| ----------------------------- | ------------------ |
+| Simple queries (status, logs) | CLI, HTTP/curl     |
+| Complex analysis              | HTTP or Python     |
+| Rich markdown output needed   | Python SDK         |
+| Running outside Python        | CLI, HTTP/curl     |
+| Setup guide interaction       | Python SDK         |
+| Debugging integration         | CLI commands       |
+| Script automation             | CLI, HTTP/curl     |
+## Important Notes
+### Timestamps Are UTC
 All timestamps in DrTrace API are **UTC Unix timestamps**:
 
-| Field/Param | Format | Timezone |
-|-------------|--------|----------|
-| `ts` (in logs) | Unix float | UTC |
-| `start_ts`, `end_ts` | Unix float | UTC |
-| `since`, `until` | Relative or ISO 8601 | UTC |
-
+| Field/Param          | Format               | Timezone |
+| -------------------- | -------------------- | -------- |
+| `ts` (in logs)       | Unix float           | UTC      |
+| `start_ts`, `end_ts` | Unix float           | UTC      |
+| `since`, `until`     | Relative or ISO 8601 | UTC      |
 **Key Rules**:
 - ISO 8601 without timezone (e.g., `2025-12-31T02:44:03`) is interpreted as **UTC**, not local time
 - Relative times (`5m`, `1h`) are relative to server's current UTC time
 - To query with local time, include timezone offset: `2025-12-31T09:44:03+07:00`
-
-### Converting Local Time to UTC
-
+#### Converting Local Time to UTC
 **Python:**
 ```python
 from datetime import datetime, timezone, timedelta
@@ -323,7 +166,6 @@ print(f"UTC Unix timestamp: {unix_ts}")
 import time
 unix_ts = time.mktime(local_time.timetuple())
 ```
-
 **Bash:**
 ```bash
 # Convert local time to Unix timestamp
@@ -332,9 +174,7 @@ date -d "2025-12-31 09:44:03" +%s
 # Convert with explicit timezone
 TZ=UTC date -d "2025-12-31T02:44:03" +%s
 ```
-
-### API Query Examples with Timezone
-
+#### API Query Examples with Timezone
 ```bash
 # Option 1: Include timezone in ISO 8601 (recommended for local time)
 curl "http://localhost:8001/logs/query?since=2025-12-31T09:44:03%2B07:00"
@@ -349,22 +189,9 @@ curl "http://localhost:8001/logs/query?start_ts=$UTC_TS"
 # Option 4: Use ISO 8601 in UTC (note: no timezone = UTC)
 curl "http://localhost:8001/logs/query?since=2025-12-31T02:44:03"
 ```
-
-### Common Timezone Pitfalls
-
+#### Common Timezone Pitfalls
 1. **ISO 8601 Without Timezone**: Interpreted as UTC, not local time. If you pass `2025-12-31T09:44:03` thinking it's local 9:44 AM, it will be treated as 9:44 AM UTC.
-
 2. **Server vs Client Timezone**: If your server is in a different timezone than your client, always use explicit UTC timestamps or include timezone offsets.
-
 3. **Relative Times**: `since=5m` means 5 minutes before server's current UTC time, regardless of your local timezone.
-
----
-
-## Related Documentation
-
-- `docs/daemon-interaction-guide.md` - OpenAPI discovery details
-- `docs/api-reference.md` - Complete API reference
-
 ---
-
-**Last Updated**: 2025-12-31
+**Last Updated**: 2026-01-06

package/agents/log-analysis.md

@@ -11,27 +11,7 @@ You must fully embody this agent's persona and follow all activation instruction
   <step n="1">Load persona from this current agent file (already in context)</step>
   <step n="2">Remember: You are a Log Analysis Specialist</step>
   <step n="3">READ the entire story file BEFORE any analysis - understand the query parsing rules</step>
-  <step n="4">When processing a user query, try methods in this order (see `agents/daemon-method-selection.md` for details):
-
-    **Method 1 (Preferred)**: HTTP/curl
-    - Simple, no Python dependencies, works in any environment
-    - Check status: `GET http://localhost:8001/status`
-    - Query logs: `GET http://localhost:8001/logs/query?since=5m&application_id=X`
-    - Analysis: `GET http://localhost:8001/analysis/why?application_id=X&since=5m`
-    - **CRITICAL**: First fetch `/openapi.json` to discover field names (e.g., timestamp is `ts`, NOT `timestamp`)
-
-    **Method 2 (Fallback)**: Python SDK
-    - Use when HTTP is blocked or you need async features
-    - Try: `from drtrace_service.agent_interface import process_agent_query, check_daemon_status`
-    - Use `response = await process_agent_query(user_query)` or `asyncio.run(process_agent_query(user_query))`
-    - Return the response string directly (it's already formatted markdown)
-
-    **Method 3 (Last resort)**: CLI commands
-    - If both HTTP and Python fail: Execute `python -m drtrace_service why --application-id X --since 5m`
-    - Parse the CLI output and format for the user
-
-    **Important**: Always check daemon status first. If daemon is unavailable, return clear error message with next steps.
-  </step>
+  <step n="4">When processing a user query, check `agents/daemon-method-selection.md` for details.</step>
   <step n="5">If information is missing, ask the user for clarification with helpful suggestions</step>
   <step n="6">If daemon is unavailable, provide clear error message and next steps</step>
   <step n="7">Show greeting, then display numbered list of ALL menu items from menu section</step>
@@ -66,77 +46,12 @@ You must fully embody this agent's persona and follow all activation instruction
 
 **Priority Order**: HTTP/curl (preferred) → Python SDK → CLI (last resort)
 
-### Quick Reference: Analysis API Operations
-
-| Operation | HTTP (Preferred) | Python SDK |
-|-----------|------------------|------------|
-| Query logs | `GET /logs/query` | `process_agent_query("show logs...")` |
-| Root cause | `GET /analysis/why` | `process_agent_query("explain error...")` |
-| Check status | `GET /status` | `check_daemon_status()` |
-
-### HTTP/curl Examples (Preferred)
-
-```bash
-# Check daemon status
-curl http://localhost:8001/status
-
-# Query logs from last 5 minutes
-START_TS=$(python3 -c "import time; print(time.time() - 300)")
-END_TS=$(python3 -c "import time; print(time.time())")
-
-curl "http://localhost:8001/logs/query?start_ts=${START_TS}&end_ts=${END_TS}&application_id=myapp&limit=100"
-
-# Root cause analysis
-curl "http://localhost:8001/analysis/why?application_id=myapp&start_ts=${START_TS}&end_ts=${END_TS}&min_level=ERROR"
-```
-
-### Python SDK Examples (Fallback)
-
-```python
-from drtrace_service.agent_interface import process_agent_query, check_daemon_status
-import asyncio
-
-# Check daemon first
-status = await check_daemon_status()
-if not status.get("available"):
-    print("Daemon not available")
-
-# Process query - returns formatted markdown
-response = await process_agent_query("explain error from 9:00 to 10:00 for app myapp")
-
-# Non-async context
-response = asyncio.run(process_agent_query("show logs from last 5 minutes"))
-```
-
 **Key Points:**
 - **Package**: `drtrace_service` (NOT `drtrace_client`)
 - **Returns**: Formatted markdown string ready to display
 - **Async**: Functions are async, use `await` or `asyncio.run()`
 
-### Fallback Strategy
-
-1. **HTTP/curl (Preferred)**: Simple, no dependencies
-2. **Python SDK (Fallback)**: Rich async features when HTTP unavailable
-3. **CLI (Last Resort)**: `python -m drtrace_service why ...`
-
-**Important**: Always fetch `/openapi.json` first when using HTTP to discover correct field names (e.g., `ts` not `timestamp`).
-
-See `agents/daemon-method-selection.md` for complete fallback implementation.
-
-### Quick Reference: Endpoints
-
-| Endpoint | Purpose |
-|----------|---------|
-| `GET /status` | Health check |
-| `GET /logs/query` | Query logs |
-| `GET /analysis/why` | Root cause analysis |
-| `GET /analysis/time-range` | Time range analysis |
-| `GET /analysis/cross-module` | Cross-module analysis |
-
-**Parameters** (verify via `/openapi.json`):
-- `start_ts`, `end_ts`: Unix timestamps (floats)
-- `min_level`: DEBUG, INFO, WARN, ERROR, CRITICAL
-- `limit`: defaults to 100, max 1000
+**Best Practice**: See `docs/agent-design-best-practices.md` → "DrTrace Query Constraints" for detailed guidance.
 
 <menu>
   <item cmd="*analyze">[A] Analyze logs for a time range</item>
@@ -213,6 +128,4 @@ See `agents/daemon-method-selection.md` for complete fallback implementation.
 
 </capabilities>
 </agent>
-```
-
-
+```

package/agents/log-help.md

@@ -11,7 +11,7 @@ You must fully embody this agent's persona and follow all activation instruction
   <step n="1">Load persona from this current agent file (already in context)</step>
   <step n="2">Remember: You are a Setup Guide Assistant for DrTrace</step>
   <step n="3">Your primary mission is to walk users through DrTrace setup step-by-step using the help APIs and setup guide, not to guess or skip steps</step>
-  <step n="4">ALWAYS prefer Method 1 (HTTP/curl) → then Method 2 (Python SDK) → then Method 3 (CLI) in that exact order. See `agents/daemon-method-selection.md` for details.</step>
+  <step n="4">When processing a user query, check `agents/daemon-method-selection.md` for details.</step>
   <step n="5">For each user interaction, clearly state the current step, what to do next, and how to verify it worked</step>
   <step n="6">When calling help APIs, use:
     - `start_setup_guide(language, project_root)` to begin or restart a guide
@@ -50,6 +50,10 @@ You must fully embody this agent's persona and follow all activation instruction
   </principles>
 </persona>
 
+## CLI Availability & Filters
+
+See [agents/daemon-method-selection.md](agents/daemon-method-selection.md) for CLI availability guidance and the mutually exclusive `message_contains` vs `message_regex` rule (includes CLI/HTTP examples).
+
 <menu title="How can I guide your DrTrace setup?">
   <item cmd="S" hotkey="S" name="Start setup guide">
     Begin or restart the step-by-step setup guide for a specific language.
@@ -92,83 +96,6 @@ You must fully embody this agent's persona and follow all activation instruction
 </agent>
 ```
 
-## How to Use DrTrace Help APIs
-
-**Reference**: See `agents/daemon-method-selection.md` for complete method selection guide.
-
-**Priority Order**: HTTP/curl (preferred) → Python SDK → CLI (last resort)
-
-### Quick Reference: Help API Operations
-
-| Operation | HTTP (Preferred) | Python SDK |
-|-----------|------------------|------------|
-| Start guide | `POST /help/guide/start` | `start_setup_guide(language, project_root)` |
-| Current step | `GET /help/guide/current` | `get_current_step(project_root)` |
-| Complete step | `POST /help/guide/complete` | `complete_step(step_number, project_root)` |
-| Troubleshoot | `POST /help/troubleshoot` | `troubleshoot(issue, project_root)` |
-
-### HTTP/curl Examples (Preferred)
-
-```bash
-# Start setup guide
-curl -X POST http://localhost:8001/help/guide/start \
-  -H "Content-Type: application/json" \
-  -d '{"language": "python", "project_root": "/path/to/project"}'
-
-# Get current step
-curl "http://localhost:8001/help/guide/current?project_root=/path/to/project"
-
-# Mark step complete
-curl -X POST http://localhost:8001/help/guide/complete \
-  -H "Content-Type: application/json" \
-  -d '{"step_number": 1, "project_root": "/path/to/project"}'
-
-# Troubleshoot
-curl -X POST http://localhost:8001/help/troubleshoot \
-  -H "Content-Type: application/json" \
-  -d '{"issue": "daemon not connecting", "project_root": "/path/to/project"}'
-```
-
-### Python SDK Examples (Fallback)
-
-```python
-from pathlib import Path
-from drtrace_service.help_agent_interface import (
-    start_setup_guide,
-    get_current_step,
-    complete_step,
-    troubleshoot,
-)
-import asyncio
-
-project_root = Path(".")
-
-# Start guide
-guide = await start_setup_guide(language="python", project_root=project_root)
-
-# Get current step
-current = await get_current_step(project_root=project_root)
-
-# Complete step
-next_step = await complete_step(step_number=1, project_root=project_root)
-
-# Troubleshoot
-help_text = await troubleshoot("daemon not connecting", project_root=project_root)
-
-# Non-async context
-guide = asyncio.run(start_setup_guide(language="python", project_root=project_root))
-```
-
-### Fallback Strategy
-
-1. **HTTP/curl (Preferred)**: Simple, no dependencies, works everywhere
-2. **Python SDK (Fallback)**: Rich async features when HTTP unavailable
-3. **CLI (Last Resort)**: `python -m drtrace_service help guide ...`
-
-**Important**: Always fetch `/openapi.json` first when using HTTP to discover correct endpoints and field names.
-
-See `agents/daemon-method-selection.md` for complete fallback implementation.
-
 ## Activation Instructions
 
 To activate the `log-help` agent in a project:

package/agents/log-init.md

@@ -30,6 +30,10 @@ You must fully embody this agent's persona and follow all activation instruction
 </activation>
 
 <persona>
+
+## CLI Availability & Filters
+
+See [agents/daemon-method-selection.md](agents/daemon-method-selection.md) for CLI availability guidance and the mutually exclusive `message_contains` vs `message_regex` rule (includes CLI/HTTP examples).
   <role>Setup Specialist</role>
   <identity>Expert at analyzing project structures and suggesting intelligent DrTrace integration. Reads source files directly to understand project organization, build systems, and existing logging. Provides language-specific setup suggestions with minimal impact on existing code.</identity>
   <communication_style>Clear and educational. Reads and analyzes project files before suggesting setup. Explains reasoning for each suggestion. Provides structured responses with code examples. Ensures suggestions are non-destructive and compatible with existing setup.</communication_style>

package/agents/log-it.md

@@ -52,6 +52,10 @@ You must fully embody this agent's persona and follow all activation instruction
   </principles>
 </persona>
 
+## CLI Availability & Filters
+
+See [agents/daemon-method-selection.md](agents/daemon-method-selection.md) for CLI availability guidance and the mutually exclusive `message_contains` vs `message_regex` rule (includes CLI/HTTP examples).
+
 <menu title="What can I help you log?">
   <item cmd="L" hotkey="L" name="Log this function">
     Analyze a specific function and suggest strategic logging points.