fuju-trace 0.1.9__tar.gz

fuju_trace-0.1.9/.gitignore

@@ -0,0 +1,41 @@
+# Rust 构建产物（巨大，绝不入库）
+target/
+**/target/
+
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+venv/
+.pytest_cache/
+
+# Node / TypeScript
+node_modules/
+dist/
+*.tsbuildinfo
+fuju-trace-console/.test-dist/
+fuju-trace-node/artifacts/
+
+# C / 扩展构建产物（tracevault-extension）
+*.o
+*.so
+*.a
+*.node
+
+# 运行时/测试落盘产物（WAL / 段 / manifest / 向量文件，正常写在 temp_dir，这里兜底）
+*.wal
+*.vortex
+manifest.dat
+vectors.dat
+
+# 编辑器 / 操作系统
+.DS_Store
+*.swp
+*~
+.idea/
+.vscode/
+.gstack/
+
+# 控制台前端构建产物（vite build 后拷入，编译期内嵌）
+fuju-trace-engine/crates/fuju-trace-engine/console_dist/

fuju_trace-0.1.9/PKG-INFO

@@ -0,0 +1,81 @@
+Metadata-Version: 2.5
+Name: fuju-trace
+Version: 0.1.9
+Summary: Fuju Trace instrumentation SDK — emit trace events with deterministic, engine-matching event_id
+Project-URL: Homepage, https://github.com/vibeinging/fuju-trace
+Project-URL: Repository, https://github.com/vibeinging/fuju-trace
+Project-URL: Issues, https://github.com/vibeinging/fuju-trace/issues
+Author: Fuju Trace
+License-Expression: MIT
+Keywords: agent,llm,observability,opentelemetry,trace,tracing
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: System :: Monitoring
+Requires-Python: >=3.8
+Provides-Extra: vexdb
+Requires-Dist: fuju-trace-vexdb[driver]==0.1.9; extra == 'vexdb'
+Description-Content-Type: text/markdown
+
+# Fuju Trace Python SDK
+
+Python 3.8+ SDK for Agent traces. It emits deterministic events and writes them through an Exporter. The base package uses only the Python standard library.
+
+## VexDB
+
+```bash
+pip install 'fuju-trace[vexdb]==0.1.9'
+```
+
+```python
+import os
+from fuju_trace import DbExporter, Tracer, connect
+
+with connect(vexdb_dsn=os.environ["VEXDB_DSN"], tenant_id=1,
+             vector_dim=3, initialize=True) as db:
+    tracer = Tracer(exporter=DbExporter(db, tenant_id=1), node_id=1)
+    with tracer.trace("risk review", tenant_id=1) as trace:
+        with trace.span("investigate") as span:
+            span.log("suspicious transaction")
+    tracer.close()
+    print(db.search(text="transaction", k=10))
+```
+
+The VexDB adapter also accepts `vexdb_params={...}`. `vector_dim` must match the embedding model; text search works without supplying vectors. For batching, use `BufferedDbExporter` and call `flush()` when reads need to see all queued writes. See [the adapter guide](../../fuju-trace-vexdb/README.md).
+
+## Local embedded DB
+
+```bash
+python -m pip install -e ./fuju-trace-sdk/python -e ./fuju-trace-db-python
+```
+
+Use `connect(path="./trace-data", tenant_id=1)` with the same `DbExporter` and `Tracer` code. The separate `fuju-trace-db` wheel contains the Rust engine.
+
+## Logging and custom sinks
+
+`ConsoleExporter` prints JSON events for local debugging. `CollectingExporter` stores events in memory for tests. `BatchExporter` wraps a sink and forwards full batches. Implement `Exporter.export_batch()` to integrate another database.
+
+```python
+from fuju_trace import ConsoleExporter, Tracer
+
+tracer = Tracer(exporter=ConsoleExporter(), node_id=1, agent_name="planner")
+with tracer.trace("request", tenant_id=1) as trace:
+    with trace.span("planner.route", display_name="Plan next step") as span:
+        span.log("ready")
+```
+
+`name` becomes the stored `span_name`; `display_name` is optional. `agent_name` identifies the actor. `event_id` is derived from `ext_span_id`, `seq`, and `event_type` and matches the Rust and TypeScript implementations.
+
+## Tests
+
+```bash
+python tests/test_sdk.py
+python -m unittest discover -s tests
+```

fuju_trace-0.1.9/README.md

@@ -0,0 +1,55 @@
+# Fuju Trace Python SDK
+
+Python 3.8+ SDK for Agent traces. It emits deterministic events and writes them through an Exporter. The base package uses only the Python standard library.
+
+## VexDB
+
+```bash
+pip install 'fuju-trace[vexdb]==0.1.9'
+```
+
+```python
+import os
+from fuju_trace import DbExporter, Tracer, connect
+
+with connect(vexdb_dsn=os.environ["VEXDB_DSN"], tenant_id=1,
+             vector_dim=3, initialize=True) as db:
+    tracer = Tracer(exporter=DbExporter(db, tenant_id=1), node_id=1)
+    with tracer.trace("risk review", tenant_id=1) as trace:
+        with trace.span("investigate") as span:
+            span.log("suspicious transaction")
+    tracer.close()
+    print(db.search(text="transaction", k=10))
+```
+
+The VexDB adapter also accepts `vexdb_params={...}`. `vector_dim` must match the embedding model; text search works without supplying vectors. For batching, use `BufferedDbExporter` and call `flush()` when reads need to see all queued writes. See [the adapter guide](../../fuju-trace-vexdb/README.md).
+
+## Local embedded DB
+
+```bash
+python -m pip install -e ./fuju-trace-sdk/python -e ./fuju-trace-db-python
+```
+
+Use `connect(path="./trace-data", tenant_id=1)` with the same `DbExporter` and `Tracer` code. The separate `fuju-trace-db` wheel contains the Rust engine.
+
+## Logging and custom sinks
+
+`ConsoleExporter` prints JSON events for local debugging. `CollectingExporter` stores events in memory for tests. `BatchExporter` wraps a sink and forwards full batches. Implement `Exporter.export_batch()` to integrate another database.
+
+```python
+from fuju_trace import ConsoleExporter, Tracer
+
+tracer = Tracer(exporter=ConsoleExporter(), node_id=1, agent_name="planner")
+with tracer.trace("request", tenant_id=1) as trace:
+    with trace.span("planner.route", display_name="Plan next step") as span:
+        span.log("ready")
+```
+
+`name` becomes the stored `span_name`; `display_name` is optional. `agent_name` identifies the actor. `event_id` is derived from `ext_span_id`, `seq`, and `event_type` and matches the Rust and TypeScript implementations.
+
+## Tests
+
+```bash
+python tests/test_sdk.py
+python -m unittest discover -s tests
+```

fuju_trace-0.1.9/fuju_trace/__init__.py

@@ -0,0 +1,45 @@
+"""fuju_trace SDK —— 给 Agent 打点，产出与 Fuju Trace 引擎一致的 trace 事件。
+
+核心保证：event_id 与引擎逐字节一致（同一套 FNV 哈希），所以 SDK 产生的事件灌进引擎后，
+去重、崩溃重放幂等全都对得上。
+"""
+from ._snowflake import Snowflake
+from .client import connect
+from .event import EventType, SpanEvent, event_id
+from .exporter import (
+    BatchExporter,
+    BufferedDbExporter,
+    CollectingExporter,
+    ConsoleExporter,
+    DbExporter,
+    Exporter,
+    NoopExporter,
+    SpoolConsumer,
+    SpoolDbExporter,
+)
+from .service import FujuTraceRuntime, get_fuju_trace_runtime, init_fuju_trace, shutdown_fuju_trace
+from .tracer import Span, Trace, Tracer
+
+__all__ = [
+    "Snowflake",
+    "connect",
+    "EventType",
+    "SpanEvent",
+    "event_id",
+    "Exporter",
+    "ConsoleExporter",
+    "CollectingExporter",
+    "DbExporter",
+    "BufferedDbExporter",
+    "SpoolDbExporter",
+    "SpoolConsumer",
+    "BatchExporter",
+    "NoopExporter",
+    "FujuTraceRuntime",
+    "init_fuju_trace",
+    "shutdown_fuju_trace",
+    "get_fuju_trace_runtime",
+    "Tracer",
+    "Trace",
+    "Span",
+]

fuju_trace-0.1.9/fuju_trace/_snowflake.py

@@ -0,0 +1,55 @@
+"""提交单调的雪花 ID 生成器。
+
+id = 41 位毫秒时间戳 | 10 位节点 | 12 位序列  → 单调、可排序、跨进程不撞。
+调度层正确性硬前置:event_id 必须真单调(不能用 SEQUENCE CACHE),这里满足。
+"""
+from __future__ import annotations
+
+import os
+import threading
+import time
+
+_EPOCH_MS = 1_577_836_800_000  # 2020-01-01Z,缩短数值
+_nodes_lock = threading.Lock()
+_nodes: dict[int, _NodeState] = {}
+
+
+class _NodeState:
+    """同一进程中，同一个 node_id 的所有 Tracer 共用计数器。"""
+
+    def __init__(self) -> None:
+        self.lock = threading.Lock()
+        self.last_ms = -1
+        self.seq = 0
+
+
+def _state_for(node: int) -> _NodeState:
+    with _nodes_lock:
+        if node not in _nodes:
+            _nodes[node] = _NodeState()
+        return _nodes[node]
+
+
+class Snowflake:
+    def __init__(self, node_id: int | None = None):
+        if node_id is None:
+            # 默认用 PID 低 10 位;多机部署应显式配不同 node_id
+            node_id = os.getpid() & 0x3FF
+        if isinstance(node_id, bool) or not isinstance(node_id, int) or not 0 <= node_id <= 0x3FF:
+            raise ValueError("node_id must be an integer from 0 to 1023")
+        self.node = node_id
+        self._state = _state_for(node_id)
+
+    def next(self) -> int:
+        state = self._state
+        with state.lock:
+            ms = max(int(time.time() * 1000), state.last_ms)
+            if ms == state.last_ms:
+                state.seq = (state.seq + 1) & 0xFFF
+                if state.seq == 0:  # 同毫秒序列耗尽,自旋到下一毫秒
+                    while ms <= state.last_ms:
+                        ms = int(time.time() * 1000)
+            else:
+                state.seq = 0
+            state.last_ms = ms
+            return ((ms - _EPOCH_MS) << 22) | (self.node << 12) | state.seq

fuju_trace-0.1.9/fuju_trace/cli.py

@@ -0,0 +1,48 @@
+"""Fuju Trace SDK command line helpers."""
+from __future__ import annotations
+
+import argparse
+import time
+from typing import Sequence
+
+from .client import connect
+from .exporter import SpoolConsumer
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(prog="fuju-trace")
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    consume = sub.add_parser("consume-spool", help="consume local spool files into an embedded Fuju Trace DB")
+    consume.add_argument("--data-dir", required=True, help="embedded Fuju Trace data directory")
+    consume.add_argument("--spool-dir", required=True, help="spool directory written by SpoolDbExporter")
+    consume.add_argument("--tenant-id", help="default tenant id when spool files do not carry one")
+    consume.add_argument("--interval", type=float, default=0.5, help="sleep seconds between polling rounds")
+    consume.add_argument("--limit", type=int, help="max events to consume per polling round")
+    consume.add_argument("--once", action="store_true", help="consume one round then exit")
+    consume.add_argument("--keep-done", action="store_true", help="move consumed files to done/ instead of deleting them")
+    return parser
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    parser = build_parser()
+    args = parser.parse_args(argv)
+    if args.command == "consume-spool":
+        db = connect(path=args.data_dir, tenant_id=args.tenant_id)
+        try:
+            consumer = SpoolConsumer(db, args.spool_dir, tenant_id=args.tenant_id, keep_done=args.keep_done)
+            while True:
+                consumed = consumer.consume_once(limit=args.limit)
+                if args.once:
+                    print(f"consumed={consumed}")
+                    return 0
+                if consumed == 0:
+                    time.sleep(args.interval)
+        finally:
+            db.close()
+    parser.error("unknown command")
+    return 2
+
+
+if __name__ == "__main__":  # pragma: no cover
+    raise SystemExit(main())

fuju_trace-0.1.9/fuju_trace/client.py

@@ -0,0 +1,61 @@
+"""Open an embedded Fuju Trace store or VexDB adapter."""
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Any, Mapping
+
+TenantId = str | int
+
+
+def connect(
+    target: str | Path | None = None,
+    *,
+    path: str | Path | None = None,
+    data_dir: str | Path | None = None,
+    vexdb_dsn: str | None = None,
+    vexdb_params: Mapping[str, Any] | None = None,
+    tenant_id: TenantId | None = None,
+    **options: Any,
+) -> Any:
+    """Open a Fuju Trace connection.
+
+    Use ``connect(path="./data")`` for an embedded DB when ``fuju-trace-db`` is
+    installed, or ``connect(vexdb_params={...}, tenant_id=..., vector_dim=...)``
+    when the optional ``fuju-trace-vexdb`` package is installed. A VexDB DSN
+    can also be supplied with ``vexdb_dsn``.
+    """
+
+    if vexdb_dsn is not None or vexdb_params is not None:
+        if target is not None or path is not None or data_dir is not None:
+            raise ValueError("connect accepts VexDB or path, not a combination")
+        if vexdb_dsn is not None and vexdb_params is not None:
+            raise ValueError("connect accepts vexdb_dsn or vexdb_params, not both")
+        if tenant_id is None or "vector_dim" not in options:
+            raise ValueError("connect(VexDB) requires tenant_id and vector_dim")
+        try:
+            from fuju_trace_vexdb import VexDBTraceStore
+        except ImportError as err:
+            raise RuntimeError("connect(VexDB) requires fuju-trace-vexdb; install with pip install 'fuju-trace[vexdb]'") from err
+        return VexDBTraceStore.open(vexdb_dsn, tenant_id=tenant_id,
+                                    connection_params=vexdb_params, **options)
+
+    if target is not None:
+        if path is not None or data_dir is not None:
+            raise ValueError("connect received both target and path/data_dir")
+        path = target
+        if str(path).startswith(("http://", "https://")):
+            raise ValueError("HTTP connections are no longer supported; use path or VexDB")
+    local_path = data_dir if data_dir is not None else path
+    if local_path is None:
+        raise ValueError("connect requires path or VexDB connection parameters")
+    try:
+        from fuju_trace_db import FujuTraceDB
+    except ImportError as err:
+        raise RuntimeError(
+            "connect(path=...) requires the embedded DB package. "
+            "Install it with: pip install fuju-trace-db. "
+        ) from err
+    return FujuTraceDB.open(local_path, tenant_id=tenant_id, **options)
+
+
+__all__ = ["connect"]

fuju_trace-0.1.9/fuju_trace/event.py

@@ -0,0 +1,104 @@
+"""事件模型 + 确定性 event_id。
+
+关键：event_id 与 Rust 引擎 `fuju-trace-core::event` **逐字节一致**（同一套 FNV-1a 哈希、同样的字段顺序）。
+这样 SDK 这边算出的 event_id 和引擎那边算出的相同 —— 同一条 span 事件无论重传几次、跨 SDK/引擎，
+去重都对得上，崩溃重放也幂等。基准值见引擎 `cargo run -p fuju-trace-core --example print_event_id`。
+"""
+from __future__ import annotations
+
+import enum
+from dataclasses import dataclass, field
+
+_MASK = 0xFFFFFFFFFFFFFFFF
+_FNV_OFFSET = 0xCBF29CE484222325
+_FNV_PRIME = 0x100000001B3
+
+
+class EventType(enum.Enum):
+    """事件类型。tag 进 event_id 哈希，必须与引擎 `EventType::tag()` 完全一致、永不改。"""
+
+    SPAN_START = 1
+    SPAN_END = 2
+    ATTR = 3
+    LOG = 4
+    ERROR = 5
+
+    def tag(self) -> int:
+        return self.value  # 1..5，与 Rust 对齐
+
+
+def _fnv1a64(data: bytes) -> int:
+    h = _FNV_OFFSET
+    for b in data:
+        h ^= b
+        h = (h * _FNV_PRIME) & _MASK
+    return h
+
+
+def event_id(ext_span_id: str, seq: int, event_type: EventType) -> int:
+    """= fnv1a64(ext_span_id(utf-8) ++ seq(8字节小端) ++ [type_tag])。与引擎逐字节一致。"""
+    data = ext_span_id.encode("utf-8") + (seq & _MASK).to_bytes(8, "little") + bytes([event_type.tag()])
+    return _fnv1a64(data)
+
+
+@dataclass
+class SpanEvent:
+    """一个 span 事件 = 引擎 WalRecord 的 SDK 侧对应物。"""
+
+    trace_id: int
+    span_id: int
+    ts: int  # 纳秒
+    seq: int  # 上报序：客户端给，原样进引擎，绝不被引擎重补
+    event_type: EventType
+    ext_span_id: str  # 跨进程稳定的 span 身份（进 event_id）
+    parent_span_id: int | None = None  # 父 span（trace 是棵树）
+    status: int | None = None
+    duration_ns: int | None = None
+    input_tokens: int | None = None  # LLM 输入 token（成本核心）
+    output_tokens: int | None = None
+    cache_read_tokens: int | None = None  # input_tokens 中由供应商缓存读取的部分
+    cache_write_tokens: int | None = None  # input_tokens 中新写入供应商缓存的部分
+    session_id: int | None = None  # 会话 id（多轮对话/agent 会话，串起多条 trace）
+    tenant_id: int | None = None  # 租户 id（逻辑隔离维度；多租户共享索引、查询强制按 tenant 过滤）
+    span_name: str | None = None  # SDK 的 span(name)，技术操作名；只在 SPAN_START 上报
+    display_name: str | None = None  # 可选前端展示名；只在 SPAN_START 上报
+    agent_name: str | None = None  # agent 名（成本/可观测按 agent 下钻）
+    tool_name: str | None = None  # 工具名（tool/function call span）
+    model: str | None = None  # 模型名（成本按模型归因）
+    input_text: str | None = None  # LLM 输入文本（prompt）—— eval 的评测上文
+    output_text: str | None = None  # LLM 输出文本（答案）—— eval 打分对象
+    logs: list[str] = field(default_factory=list)
+    attrs: dict[str, str | int | float | bool] = field(default_factory=dict)
+
+    def event_id(self) -> int:
+        return event_id(self.ext_span_id, self.seq, self.event_type)
+
+    def to_wire(self) -> dict:
+        """灌进引擎摄入端的 JSON 载荷（字段对齐引擎 WalRecord）。"""
+        return {
+            "trace_id": self.trace_id,
+            "span_id": self.span_id,
+            "ts": self.ts,
+            "seq": self.seq,
+            "event_type": self.event_type.value,
+            "ext_span_id": self.ext_span_id,
+            "parent_span_id": self.parent_span_id,
+            "event_id": self.event_id(),
+            "status": self.status,
+            "duration_ns": self.duration_ns,
+            "input_tokens": self.input_tokens,
+            "output_tokens": self.output_tokens,
+            "cache_read_tokens": self.cache_read_tokens,
+            "cache_write_tokens": self.cache_write_tokens,
+            "session_id": self.session_id,
+            "tenant_id": self.tenant_id,
+            "span_name": self.span_name,
+            "display_name": self.display_name,
+            "agent_name": self.agent_name,
+            "tool_name": self.tool_name,
+            "model": self.model,
+            "input_text": self.input_text,
+            "output_text": self.output_text,
+            "logs": list(self.logs),
+            "attrs": dict(self.attrs),
+        }