agentstracer 1.0.0__tar.gz

agentstracer-1.0.0/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2024 kaiaiagent (Original Author)
+Copyright (c) 2024 cyijun (Modified Version)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

agentstracer-1.0.0/MANIFEST.in

@@ -0,0 +1 @@
+recursive-include agentstrace/web/frontend/dist *

agentstracer-1.0.0/PKG-INFO

@@ -0,0 +1,170 @@
+Metadata-Version: 2.4
+Name: agentstracer
+Version: 1.0.0
+Summary: Review, score, and curate your coding agent conversation traces locally
+Author: kaiaiagent
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/cyijun/agentstrace
+Project-URL: Repository, https://github.com/cyijun/agentstrace
+Keywords: claude-code,codex,gemini-cli,opencode,openclaw,dataset,conversations
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Software Development :: Libraries
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: dev
+Requires-Dist: pytest; extra == "dev"
+Dynamic: license-file
+
+# AgentsTrace (Local-Only Fork)
+
+原版：[kaiaiagent/clawtrace](https://github.com/kaiaiagent/clawtrace)
+
+---
+
+## 快速开始
+
+### 安装
+
+```bash
+git clone https://github.com/cyijun/agentstrace.git
+cd agentstrace-local
+pip install -e .
+```
+
+### 基本使用
+
+```bash
+# 1. 配置导出源（claude/kimi/codex/gemini/all）
+agentstrace config --source all
+
+# 2. 导出对话记录
+agentstrace export --no-push -o my_conversations.jsonl
+```
+
+### 私人使用（保留 API Keys）
+
+```bash
+# 禁用 secrets 脱敏（仅建议本地私人使用）
+agentstrace config --no-secrets-redaction
+
+# 导出（包含原始 API keys）
+agentstrace export --no-push -o my_data.jsonl
+
+# ⚠️ 警告：此文件包含明文 API keys，请勿分享！
+```
+
+### 其他命令
+
+```bash
+# 列出发现的项目
+agentstrace list
+
+# 启动本地 Web UI
+agentstrace serve
+
+# 查看配置
+agentstrace config
+```
+
+---
+
+## 支持的 AI 工具
+
+| 工具 | 数据位置 | 状态 |
+|------|---------|------|
+| Claude Code | `~/.claude/projects/` | ✅ |
+| Kimi CLI | `~/.kimi/sessions/` | ✅ |
+| Codex CLI | `~/.codex/sessions/` | ✅ |
+| OpenCode | `~/.local/share/opencode/` | ✅ |
+| OpenClaw | `~/.openclaw/` | ✅ |
+| Gemini CLI | `~/.gemini/tmp/` | ✅ |
+
+---
+
+## 导出格式
+
+**JSONL**（每行一个 JSON 对象）：
+
+```jsonl
+{"session_id": "abc-123", "model": "kimi-k2", "messages": [...], ...}
+{"session_id": "def-456", "model": "claude-3-7", "messages": [...], ...}
+```
+
+### 主要字段
+
+| 字段 | 说明 |
+|------|------|
+| `session_id` | 会话唯一标识 |
+| `model` | AI 模型名称 |
+| `project` | 项目名称（已脱敏） |
+| `source` | 来源（claude/kimi/codex等） |
+| `start_time` / `end_time` | ISO 8601 时间 |
+| `messages` | 对话消息列表 |
+| `stats` | 统计信息 |
+
+### messages 结构
+
+```json
+{
+  "role": "user|assistant",
+  "content": "消息内容",
+  "thinking": "思考过程（assistant）",
+  "timestamp": "2024-01-01T12:00:00Z",
+  "tool_uses": [{
+    "tool": "bash",
+    "input": {"command": "ls -la"},
+    "output": {"text": "..."},
+    "status": "success"
+  }]
+}
+```
+
+---
+
+## 关于本版本
+
+这是 [AgentsTrace](https://github.com/kaiaiagent/clawtrace) 的修改版本，专注于**本地隐私保护**和**纯离线使用**。
+
+### 主要修改点
+
+#### 1. 移除所有网络功能
+- ❌ 云上传功能
+- ❌ Skill 下载功能
+- ❌ 浏览器自动打开
+- ❌ 所有 urllib 网络请求
+
+#### 2. 可选禁用 Secrets 脱敏
+- 新增 `--no-secrets-redaction` 配置
+- 私人使用时保留 API keys
+- 路径/用户名脱敏始终启用
+
+#### 3. 安全审查
+- ✅ 无 `eval()` / `exec()` / `compile()`
+- ✅ 无动态代码执行
+- ✅ 无反序列化风险
+- ✅ 路径遍历已防护
+
+### 与原版对比
+
+| 功能 | 原版 | 本版本 |
+|------|------|--------|
+| 云上传 | ✅ | ❌ 已移除 |
+| Skill 下载 | ✅ | ❌ 已移除 |
+| 可选禁用脱敏 | ❌ | ✅ 支持 |
+| 适用场景 | 分享数据集 | 私人本地分析 |
+
+---
+
+## 许可证
+
+MIT License - 详见 [LICENSE](LICENSE)
+
+Copyright (c) 2024 kaiaiagent (Original Author)  
+Copyright (c) 2024 cyijun (Modified Version)

agentstracer-1.0.0/README.md

@@ -0,0 +1,146 @@
+# AgentsTrace (Local-Only Fork)
+
+原版：[kaiaiagent/clawtrace](https://github.com/kaiaiagent/clawtrace)
+
+---
+
+## 快速开始
+
+### 安装
+
+```bash
+git clone https://github.com/cyijun/agentstrace.git
+cd agentstrace-local
+pip install -e .
+```
+
+### 基本使用
+
+```bash
+# 1. 配置导出源（claude/kimi/codex/gemini/all）
+agentstrace config --source all
+
+# 2. 导出对话记录
+agentstrace export --no-push -o my_conversations.jsonl
+```
+
+### 私人使用（保留 API Keys）
+
+```bash
+# 禁用 secrets 脱敏（仅建议本地私人使用）
+agentstrace config --no-secrets-redaction
+
+# 导出（包含原始 API keys）
+agentstrace export --no-push -o my_data.jsonl
+
+# ⚠️ 警告：此文件包含明文 API keys，请勿分享！
+```
+
+### 其他命令
+
+```bash
+# 列出发现的项目
+agentstrace list
+
+# 启动本地 Web UI
+agentstrace serve
+
+# 查看配置
+agentstrace config
+```
+
+---
+
+## 支持的 AI 工具
+
+| 工具 | 数据位置 | 状态 |
+|------|---------|------|
+| Claude Code | `~/.claude/projects/` | ✅ |
+| Kimi CLI | `~/.kimi/sessions/` | ✅ |
+| Codex CLI | `~/.codex/sessions/` | ✅ |
+| OpenCode | `~/.local/share/opencode/` | ✅ |
+| OpenClaw | `~/.openclaw/` | ✅ |
+| Gemini CLI | `~/.gemini/tmp/` | ✅ |
+
+---
+
+## 导出格式
+
+**JSONL**（每行一个 JSON 对象）：
+
+```jsonl
+{"session_id": "abc-123", "model": "kimi-k2", "messages": [...], ...}
+{"session_id": "def-456", "model": "claude-3-7", "messages": [...], ...}
+```
+
+### 主要字段
+
+| 字段 | 说明 |
+|------|------|
+| `session_id` | 会话唯一标识 |
+| `model` | AI 模型名称 |
+| `project` | 项目名称（已脱敏） |
+| `source` | 来源（claude/kimi/codex等） |
+| `start_time` / `end_time` | ISO 8601 时间 |
+| `messages` | 对话消息列表 |
+| `stats` | 统计信息 |
+
+### messages 结构
+
+```json
+{
+  "role": "user|assistant",
+  "content": "消息内容",
+  "thinking": "思考过程（assistant）",
+  "timestamp": "2024-01-01T12:00:00Z",
+  "tool_uses": [{
+    "tool": "bash",
+    "input": {"command": "ls -la"},
+    "output": {"text": "..."},
+    "status": "success"
+  }]
+}
+```
+
+---
+
+## 关于本版本
+
+这是 [AgentsTrace](https://github.com/kaiaiagent/clawtrace) 的修改版本，专注于**本地隐私保护**和**纯离线使用**。
+
+### 主要修改点
+
+#### 1. 移除所有网络功能
+- ❌ 云上传功能
+- ❌ Skill 下载功能
+- ❌ 浏览器自动打开
+- ❌ 所有 urllib 网络请求
+
+#### 2. 可选禁用 Secrets 脱敏
+- 新增 `--no-secrets-redaction` 配置
+- 私人使用时保留 API keys
+- 路径/用户名脱敏始终启用
+
+#### 3. 安全审查
+- ✅ 无 `eval()` / `exec()` / `compile()`
+- ✅ 无动态代码执行
+- ✅ 无反序列化风险
+- ✅ 路径遍历已防护
+
+### 与原版对比
+
+| 功能 | 原版 | 本版本 |
+|------|------|--------|
+| 云上传 | ✅ | ❌ 已移除 |
+| Skill 下载 | ✅ | ❌ 已移除 |
+| 可选禁用脱敏 | ❌ | ✅ 支持 |
+| 适用场景 | 分享数据集 | 私人本地分析 |
+
+---
+
+## 许可证
+
+MIT License - 详见 [LICENSE](LICENSE)
+
+Copyright (c) 2024 kaiaiagent (Original Author)  
+Copyright (c) 2024 cyijun (Modified Version)

agentstracer-1.0.0/agentstrace/__init__.py

@@ -0,0 +1,3 @@
+"""AgentsTrace — Export and manage coding agent conversation data."""
+
+__version__ = "1.0.0"

agentstracer-1.0.0/agentstrace/anonymizer.py

@@ -0,0 +1,112 @@
+"""Anonymize PII in Claude Code log data."""
+
+import hashlib
+import os
+import re
+
+
+def _hash_username(username: str) -> str:
+    return "user_" + hashlib.sha256(username.encode()).hexdigest()[:8]
+
+
+def _detect_home_dir() -> tuple[str, str]:
+    home = os.path.expanduser("~")
+    username = os.path.basename(home)
+    return home, username
+
+
+def anonymize_path(path: str, username: str, username_hash: str, home: str | None = None) -> str:
+    """Strip a path to project-relative and hash the username."""
+    if not path:
+        return path
+
+    if home is None:
+        home = os.path.expanduser("~")
+    prefixes = set()
+    for base in (f"/Users/{username}", f"/home/{username}", home):
+        for subdir in ("Documents", "Downloads", "Desktop"):
+            prefixes.add(f"{base}/{subdir}/")
+        prefixes.add(f"{base}/")
+
+    # Try longest prefixes first (subdirectory matches before bare home)
+    home_patterns = sorted(prefixes, key=len, reverse=True)
+
+    for prefix in home_patterns:
+        if path.startswith(prefix):
+            rest = path[len(prefix):]
+            if "/Documents/" in prefix or "/Downloads/" in prefix or "/Desktop/" in prefix:
+                return rest
+            return f"{username_hash}/{rest}"
+
+    path = path.replace(f"/Users/{username}/", f"/{username_hash}/")
+    path = path.replace(f"/home/{username}/", f"/{username_hash}/")
+
+    return path
+
+
+def anonymize_text(text: str, username: str, username_hash: str) -> str:
+    if not text or not username:
+        return text
+
+    escaped = re.escape(username)
+
+    # Replace /Users/<username> and /home/<username>
+    text = re.sub(rf"/Users/{escaped}(?=/|[^a-zA-Z0-9_-]|$)", f"/{username_hash}", text)
+    text = re.sub(rf"/home/{escaped}(?=/|[^a-zA-Z0-9_-]|$)", f"/{username_hash}", text)
+
+    # Catch hyphen-encoded paths: -Users-peteromalley- or -Users-peteromalley/
+    text = re.sub(rf"-Users-{escaped}(?=-|/|$)", f"-Users-{username_hash}", text)
+    text = re.sub(rf"-home-{escaped}(?=-|/|$)", f"-home-{username_hash}", text)
+
+    # Also handle underscore-to-hyphen encoding: kaid_aiagent → kaid-aiagent
+    if "_" in username:
+        hyphen_variant = username.replace("_", "-")
+        hyphen_escaped = re.escape(hyphen_variant)
+        text = re.sub(rf"-Users-{hyphen_escaped}(?=-|/|$)", f"-Users-{username_hash}", text)
+        text = re.sub(rf"-home-{hyphen_escaped}(?=-|/|$)", f"-home-{username_hash}", text)
+
+    # Catch temp paths like /private/tmp/claude-501/-Users-peteromalley/
+    text = re.sub(rf"claude-\d+/-Users-{escaped}", f"claude-XXX/-Users-{username_hash}", text)
+
+    # Final pass: replace bare username in remaining contexts (ls output, prose, etc.)
+    # Only if username is >= 4 chars to avoid false positives
+    if len(username) >= 4:
+        text = re.sub(rf"\b{escaped}\b", username_hash, text)
+
+    return text
+
+
+class Anonymizer:
+    """Stateful anonymizer that consistently hashes usernames."""
+
+    def __init__(self, extra_usernames: list[str] | None = None):
+        self.home, self.username = _detect_home_dir()
+        self.username_hash = _hash_username(self.username)
+
+        # Additional usernames to anonymize (GitHub handles, Discord names, etc.)
+        self._extra: list[tuple[str, str]] = []
+        for name in (extra_usernames or []):
+            name = name.strip()
+            if name and name != self.username:
+                self._extra.append((name, _hash_username(name)))
+
+    def path(self, file_path: str) -> str:
+        result = anonymize_path(file_path, self.username, self.username_hash, self.home)
+        result = anonymize_text(result, self.username, self.username_hash)
+        for name, hashed in self._extra:
+            result = _replace_username(result, name, hashed)
+        return result
+
+    def text(self, content: str) -> str:
+        result = anonymize_text(content, self.username, self.username_hash)
+        for name, hashed in self._extra:
+            result = _replace_username(result, name, hashed)
+        return result
+
+
+def _replace_username(text: str, username: str, username_hash: str) -> str:
+    if not text or not username or len(username) < 3:
+        return text
+    escaped = re.escape(username)
+    text = re.sub(escaped, username_hash, text, flags=re.IGNORECASE)
+    return text

agentstracer-1.0.0/agentstrace/backends.py

@@ -0,0 +1,211 @@
+"""Shared backend detection and resolution for coding-agent CLIs.
+
+Used by both the scoring pipeline and PII review to auto-detect whether
+agentstrace is running under Claude Code, Codex, or OpenClaw and dispatch
+to the corresponding automation CLI.
+"""
+
+from __future__ import annotations
+
+import os
+import shutil
+import subprocess
+from pathlib import Path
+
+
+SUPPORTED_BACKENDS = ("claude", "codex", "openclaw")
+BACKEND_CHOICES = ("auto", *SUPPORTED_BACKENDS)
+BACKEND_COMMANDS: dict[str, str] = {
+    "claude": "claude",
+    "codex": "codex",
+    "openclaw": "openclaw",
+}
+BACKEND_ENV_MARKERS: dict[str, tuple[str, ...]] = {
+    "claude": ("CLAUDECODE", "CLAUDE_CODE", "CLAUDECODE_SESSION_ID", "CLAUDE_PROJECT_DIR"),
+    "codex": ("CODEX_THREAD_ID", "CODEX_SANDBOX", "CODEX_CI"),
+    "openclaw": ("OPENCLAW_HOME", "OPENCLAW_STATE_DIR", "OPENCLAW_CONFIG_PATH"),
+}
+BACKEND_COMMAND_ALIASES: dict[str, tuple[str, ...]] = {
+    "claude": ("claude",),
+    "codex": ("codex",),
+    "openclaw": ("openclaw",),
+}
+
+
+def _detect_current_agent_from_env(env: dict[str, str] | None = None) -> str | None:
+    """Infer the current agent from the process environment."""
+    env = os.environ if env is None else env
+    for backend, keys in BACKEND_ENV_MARKERS.items():
+        for key in keys:
+            if env.get(key):
+                return backend
+    return None
+
+
+def _get_process_field(pid: int, field: str) -> str:
+    """Read a single process field from ps, returning an empty string on failure."""
+    try:
+        proc = subprocess.run(
+            ["ps", f"-o{field}=", "-p", str(pid)],
+            capture_output=True,
+            text=True,
+            timeout=2,
+        )
+    except (FileNotFoundError, subprocess.TimeoutExpired, OSError):
+        return ""
+    if proc.returncode != 0:
+        return ""
+    return proc.stdout.strip()
+
+
+def _classify_process_command(comm: str, command: str) -> str | None:
+    """Map a process command to a supported backend."""
+    fields = " ".join(part for part in (comm, command) if part).lower()
+    if not fields:
+        return None
+    base = Path(comm).name.lower() if comm else ""
+    for backend, aliases in BACKEND_COMMAND_ALIASES.items():
+        for alias in aliases:
+            if base == alias or f" {alias}" in f" {fields}" or f"/{alias}" in fields:
+                return backend
+    return None
+
+
+def _detect_current_agent_from_process_tree(pid: int | None = None, *, max_depth: int = 6) -> str | None:
+    """Walk parent processes to find a known coding-agent CLI."""
+    current_pid = pid if pid is not None else os.getppid()
+    depth = 0
+    seen: set[int] = set()
+
+    while current_pid > 1 and depth < max_depth and current_pid not in seen:
+        seen.add(current_pid)
+        comm = _get_process_field(current_pid, "comm")
+        command = _get_process_field(current_pid, "command")
+        detected = _classify_process_command(comm, command)
+        if detected:
+            return detected
+        parent_text = _get_process_field(current_pid, "ppid")
+        try:
+            current_pid = int(parent_text)
+        except ValueError:
+            break
+        depth += 1
+    return None
+
+
+def detect_current_agent(env: dict[str, str] | None = None) -> str | None:
+    """Detect the current coding agent from env vars or process tree."""
+    return _detect_current_agent_from_env(env) or _detect_current_agent_from_process_tree()
+
+
+def resolve_backend(backend: str = "auto", env: dict[str, str] | None = None) -> str:
+    """Resolve 'auto' backend selection to a concrete backend name.
+
+    Priority: explicit value > AGENTSTRACE_SCORER_BACKEND env > auto-detect.
+    """
+    env = os.environ if env is None else env
+    requested = (backend or "auto").strip().lower()
+    if requested != "auto":
+        if requested not in SUPPORTED_BACKENDS:
+            raise RuntimeError(f"Unsupported backend: {backend}")
+        return requested
+
+    override = (env.get("AGENTSTRACE_SCORER_BACKEND") or "").strip().lower()
+    if override:
+        if override not in SUPPORTED_BACKENDS:
+            raise RuntimeError(
+                f"Unsupported AGENTSTRACE_SCORER_BACKEND value: {override}. "
+                f"Use one of: {', '.join(SUPPORTED_BACKENDS)}."
+            )
+        return override
+
+    detected = detect_current_agent(env)
+    if detected:
+        return detected
+
+    raise RuntimeError(
+        "Could not detect the current agent. "
+        "Run agentstrace from a supported agent CLI, set AGENTSTRACE_SCORER_BACKEND, "
+        "or pass --backend explicitly."
+    )
+
+
+def require_backend_command(backend: str) -> str:
+    """Return the CLI command for a backend, ensuring it is installed."""
+    command = BACKEND_COMMANDS[backend]
+    if shutil.which(command) is None:
+        raise RuntimeError(f"{backend} CLI not found. Install it or choose a different --backend.")
+    return command
+
+
+def check_backend_runtime(backend: str, env: dict[str, str] | None = None) -> None:
+    """Backend-specific runtime preflight hook (extensible, currently a no-op)."""
+    _ = backend, env
+
+
+def summarize_process_error(stderr: str, stdout: str = "") -> str:
+    """Return the most actionable error line from subprocess output."""
+    lines: list[str] = []
+    for raw in f"{stderr}\n{stdout}".splitlines():
+        line = raw.strip()
+        if not line:
+            continue
+        if line.startswith("WARNING: proceeding, even though we could not update PATH"):
+            continue
+        if line.startswith("note: run with `RUST_BACKTRACE=1`"):
+            continue
+        if line.startswith("thread '"):
+            continue
+        lines.append(line)
+
+    if not lines:
+        return ""
+
+    for line in reversed(lines):
+        lower = line.lower()
+        if (
+            lower.startswith("error:")
+            or " error " in lower
+            or "failed" in lower
+            or "unauthorized" in lower
+        ):
+            return line
+
+    return lines[-1]
+
+
+def format_codex_runtime_error(returncode: int, stderr: str, stdout: str = "") -> str:
+    """Normalize common Codex exec failures into actionable guidance."""
+    combined = "\n".join(part.strip() for part in (stderr, stdout) if part and part.strip())
+    lower = combined.lower()
+
+    if (
+        "failed to lookup address information" in lower
+        or "temporary failure in name resolution" in lower
+        or "name or service not known" in lower
+        or "network is unreachable" in lower
+        or "could not resolve host" in lower
+    ):
+        return (
+            "Codex runs through `codex exec` in non-interactive mode. "
+            "This process could not reach the Codex backend from the current environment. "
+            "If you launched agentstrace inside a network-disabled Codex sandbox, "
+            "rerun it from your host shell or with network access."
+        )
+
+    if (
+        "401" in lower
+        or "unauthorized" in lower
+        or "not signed in" in lower
+        or "authentication required" in lower
+    ):
+        return (
+            "Codex runs through `codex exec` in non-interactive mode. "
+            "`codex exec` reuses saved CLI authentication by default; for automation, "
+            "run `codex login` or set `CODEX_API_KEY` before running agentstrace."
+        )
+
+    summary = summarize_process_error(stderr, stdout)
+    if summary:
+        return f"codex exited {returncode}: {summary}"
+    return f"codex exited {returncode}"