ams-observability 0.1.0__tar.gz

ams_observability-0.1.0/.github/workflows/release.yml

@@ -0,0 +1,26 @@
+name: Publish to PyPI
+
+# Publishes ams-observability to PyPI when a GitHub Release is published.
+# Uses PyPI Trusted Publishing (OIDC) — no API token is stored anywhere.
+# One-time setup on PyPI: add a trusted publisher for project
+# `ams-observability` (owner `mathu97`, repo `ams`, workflow `release.yml`).
+#
+# To cut a release: bump `version` in pyproject.toml, then create a GitHub
+# Release (tag like v0.1.0). Publishing runs automatically.
+
+on:
+  release:
+    types: [published]
+
+jobs:
+  publish:
+    runs-on: ubuntu-latest
+    permissions:
+      id-token: write  # required for trusted publishing
+    steps:
+      - uses: actions/checkout@v4
+      - uses: astral-sh/setup-uv@v5
+      - name: Build sdist and wheel
+        run: uv build
+      - name: Publish to PyPI
+        uses: pypa/gh-action-pypi-publish@release/v1

ams_observability-0.1.0/.gitignore

@@ -0,0 +1,13 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.venv/
+venv/
+.env
+.pytest_cache/
+.ruff_cache/
+ams-data/
+.DS_Store

ams_observability-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 mathu97
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

ams_observability-0.1.0/PKG-INFO

@@ -0,0 +1,180 @@
+Metadata-Version: 2.4
+Name: ams-observability
+Version: 0.1.0
+Summary: A super simple monitoring system for Claude agents — full session traces as JSON in blob storage.
+Project-URL: Homepage, https://github.com/mathu97/ams
+Project-URL: Repository, https://github.com/mathu97/ams
+Author: mathu97
+License: MIT
+License-File: LICENSE
+Keywords: agents,claude,llm,monitoring,observability,tracing
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: System :: Monitoring
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Requires-Dist: boto3>=1.28
+Requires-Dist: pydantic>=2.0
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# AMS — Agent Monitoring System
+
+A **super simple** monitoring system for [Claude agents](https://docs.claude.com/en/api/agent-sdk/overview). Capture a whole Claude Agent SDK session end to end — every tool call, every subagent and *why* it was invoked, the model's reasoning, results, timing, and cost — as **one readable JSON object** in blob storage.
+
+No collector, no database, no agent. One JSON file per session in S3-compatible storage. Built to be trivially easy to read and filter (the things that make Arize and friends painful).
+
+```python
+from ams.claude import traced_query
+
+async for message in traced_query(prompt="Cancel my membership", options=options):
+    ...
+# session written to storage automatically when the stream ends
+```
+
+That's the whole integration. Swap `query` for `traced_query`.
+
+## Why
+
+We monitor our Claude agents with Arize today, but it's hard to read, and hard to search/filter for a single session. AMS keeps the data model deliberately flat and typed so a session is obvious to a human and easy to query by a machine. Field names follow the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) (`gen_ai.*`) where there's a natural equivalent, so the data can later be re-emitted as OTLP without renaming.
+
+## What it captures
+
+A session is one trace of ordered **events**:
+
+| Event | Source | Detail captured |
+|---|---|---|
+| `user_prompt` | `UserPromptSubmit` hook | the prompt |
+| `llm_message` | message stream | model, **thinking / chain-of-thought**, assistant text, token usage |
+| `tool_call` | `PreToolUse` + `PostToolUse` / `PostToolUseFailure` hooks | tool name, input, result, error, **timing** |
+| `subagent` | `SubagentStart` / `SubagentStop` hooks | agent type, **why it was invoked** (the prompt), transcript path; child tool calls nest underneath |
+| `notification` | `Notification` hook | message |
+
+Plus session **totals**: token usage (incl. cache read/write), cost (USD), turn count, tool/subagent/error counts, and wall-clock + API duration.
+
+The Claude Agent SDK has **no built-in OpenTelemetry** — AMS captures everything through hooks and the message stream, which together are the only place this data lives.
+
+## Install
+
+```bash
+pip install ams-observability     # published name; you import it as `ams`
+```
+
+Or from a local checkout: `pip install -e .`. S3-compatible storage (boto3) is included by default.
+
+Requires Python 3.10+ and the [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk/) in your project.
+
+## Configure storage
+
+S3-compatible storage is the default. Works against **AWS S3, Cloudflare R2, MinIO** — anything speaking the S3 API.
+
+```bash
+export AMS_S3_BUCKET=my-agent-traces
+export AMS_S3_PREFIX=ams                 # optional, default "ams"
+export AMS_S3_ENDPOINT_URL=https://<account>.r2.cloudflarestorage.com   # omit for AWS S3
+export AMS_S3_REGION=auto
+# credentials via the standard AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY
+```
+
+Or write to local disk for development:
+
+```bash
+export AMS_STORAGE=local
+export AMS_LOCAL_DIR=./ams-data          # optional
+```
+
+### Layout in the bucket
+
+```
+{prefix}/sessions/{YYYY}/{MM}/{DD}/{session_id}.json   full session
+{prefix}/index/{session_id}.json                        compact summary (for listing/filtering)
+```
+
+The small `index/` objects let a frontend build a searchable session list without opening every full session.
+
+## Usage
+
+### One-call (drop-in for `query`)
+
+```python
+from ams import Agent
+from ams.claude import traced_query
+
+async for message in traced_query(
+    prompt="...",
+    options=options,                       # your ClaudeAgentOptions
+    agent=Agent(name="support-bot", version="2026.06"),
+    environment="prod",
+    tags=["voice", "cancellation"],
+    metadata={"team_id": "t_42"},
+):
+    print(message)
+```
+
+### With `ClaudeSDKClient`
+
+Merge AMS hooks into your options, feed messages to the tracer, and `finish()` when done:
+
+```python
+from ams import Tracer
+from ams.claude import instrument_options
+
+tracer = Tracer(environment="prod", tags=["chat"])
+options = instrument_options(my_options, tracer)
+
+async with ClaudeSDKClient(options=options) as client:
+    await client.query("...")
+    async for message in client.receive_response():
+        tracer.record_message(message)
+
+session = tracer.finish()
+```
+
+### Custom storage
+
+Pass any object with `put_session(session) -> str`:
+
+```python
+tracer = Tracer(storage=MyStorage())
+```
+
+## Options
+
+| Tracer arg / env | Default | Notes |
+|---|---|---|
+| `storage` / `AMS_STORAGE` | S3 | `local` to write to disk |
+| `agent` | — | `Agent(name=..., version=...)` |
+| `environment` | — | e.g. `prod`, `staging` |
+| `tags`, `metadata` | — | free-form, promoted into the index for filtering |
+| `capture_thinking` | `True` | record the model's reasoning blocks |
+| `redact` / `AMS_REDACT` | `False` | opt-in PII redaction (email / phone / card / SSN) |
+
+AMS never throws into your agent: hook and storage failures are logged, not raised.
+
+## How it works
+
+See [`docs/architecture.md`](docs/architecture.md) for the module map and the two-channel design (hooks + message stream) that AMS fuses into one session.
+
+## Schema
+
+See [`docs/schema.md`](docs/schema.md) for the full session JSON schema with an example. The contract lives in one file: [`ams/schema.py`](ams/schema.py).
+
+## Frontend
+
+A simple frontend to browse and filter sessions is planned (not built yet). See [`docs/frontend-notes.md`](docs/frontend-notes.md) for the intended design — it reads the `index/` summaries to list sessions and fetches a full session JSON on click.
+
+## Development
+
+```bash
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT

ams_observability-0.1.0/README.md

@@ -0,0 +1,156 @@
+# AMS — Agent Monitoring System
+
+A **super simple** monitoring system for [Claude agents](https://docs.claude.com/en/api/agent-sdk/overview). Capture a whole Claude Agent SDK session end to end — every tool call, every subagent and *why* it was invoked, the model's reasoning, results, timing, and cost — as **one readable JSON object** in blob storage.
+
+No collector, no database, no agent. One JSON file per session in S3-compatible storage. Built to be trivially easy to read and filter (the things that make Arize and friends painful).
+
+```python
+from ams.claude import traced_query
+
+async for message in traced_query(prompt="Cancel my membership", options=options):
+    ...
+# session written to storage automatically when the stream ends
+```
+
+That's the whole integration. Swap `query` for `traced_query`.
+
+## Why
+
+We monitor our Claude agents with Arize today, but it's hard to read, and hard to search/filter for a single session. AMS keeps the data model deliberately flat and typed so a session is obvious to a human and easy to query by a machine. Field names follow the [OpenTelemetry GenAI semantic conventions](https://opentelemetry.io/docs/specs/semconv/gen-ai/) (`gen_ai.*`) where there's a natural equivalent, so the data can later be re-emitted as OTLP without renaming.
+
+## What it captures
+
+A session is one trace of ordered **events**:
+
+| Event | Source | Detail captured |
+|---|---|---|
+| `user_prompt` | `UserPromptSubmit` hook | the prompt |
+| `llm_message` | message stream | model, **thinking / chain-of-thought**, assistant text, token usage |
+| `tool_call` | `PreToolUse` + `PostToolUse` / `PostToolUseFailure` hooks | tool name, input, result, error, **timing** |
+| `subagent` | `SubagentStart` / `SubagentStop` hooks | agent type, **why it was invoked** (the prompt), transcript path; child tool calls nest underneath |
+| `notification` | `Notification` hook | message |
+
+Plus session **totals**: token usage (incl. cache read/write), cost (USD), turn count, tool/subagent/error counts, and wall-clock + API duration.
+
+The Claude Agent SDK has **no built-in OpenTelemetry** — AMS captures everything through hooks and the message stream, which together are the only place this data lives.
+
+## Install
+
+```bash
+pip install ams-observability     # published name; you import it as `ams`
+```
+
+Or from a local checkout: `pip install -e .`. S3-compatible storage (boto3) is included by default.
+
+Requires Python 3.10+ and the [`claude-agent-sdk`](https://pypi.org/project/claude-agent-sdk/) in your project.
+
+## Configure storage
+
+S3-compatible storage is the default. Works against **AWS S3, Cloudflare R2, MinIO** — anything speaking the S3 API.
+
+```bash
+export AMS_S3_BUCKET=my-agent-traces
+export AMS_S3_PREFIX=ams                 # optional, default "ams"
+export AMS_S3_ENDPOINT_URL=https://<account>.r2.cloudflarestorage.com   # omit for AWS S3
+export AMS_S3_REGION=auto
+# credentials via the standard AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY
+```
+
+Or write to local disk for development:
+
+```bash
+export AMS_STORAGE=local
+export AMS_LOCAL_DIR=./ams-data          # optional
+```
+
+### Layout in the bucket
+
+```
+{prefix}/sessions/{YYYY}/{MM}/{DD}/{session_id}.json   full session
+{prefix}/index/{session_id}.json                        compact summary (for listing/filtering)
+```
+
+The small `index/` objects let a frontend build a searchable session list without opening every full session.
+
+## Usage
+
+### One-call (drop-in for `query`)
+
+```python
+from ams import Agent
+from ams.claude import traced_query
+
+async for message in traced_query(
+    prompt="...",
+    options=options,                       # your ClaudeAgentOptions
+    agent=Agent(name="support-bot", version="2026.06"),
+    environment="prod",
+    tags=["voice", "cancellation"],
+    metadata={"team_id": "t_42"},
+):
+    print(message)
+```
+
+### With `ClaudeSDKClient`
+
+Merge AMS hooks into your options, feed messages to the tracer, and `finish()` when done:
+
+```python
+from ams import Tracer
+from ams.claude import instrument_options
+
+tracer = Tracer(environment="prod", tags=["chat"])
+options = instrument_options(my_options, tracer)
+
+async with ClaudeSDKClient(options=options) as client:
+    await client.query("...")
+    async for message in client.receive_response():
+        tracer.record_message(message)
+
+session = tracer.finish()
+```
+
+### Custom storage
+
+Pass any object with `put_session(session) -> str`:
+
+```python
+tracer = Tracer(storage=MyStorage())
+```
+
+## Options
+
+| Tracer arg / env | Default | Notes |
+|---|---|---|
+| `storage` / `AMS_STORAGE` | S3 | `local` to write to disk |
+| `agent` | — | `Agent(name=..., version=...)` |
+| `environment` | — | e.g. `prod`, `staging` |
+| `tags`, `metadata` | — | free-form, promoted into the index for filtering |
+| `capture_thinking` | `True` | record the model's reasoning blocks |
+| `redact` / `AMS_REDACT` | `False` | opt-in PII redaction (email / phone / card / SSN) |
+
+AMS never throws into your agent: hook and storage failures are logged, not raised.
+
+## How it works
+
+See [`docs/architecture.md`](docs/architecture.md) for the module map and the two-channel design (hooks + message stream) that AMS fuses into one session.
+
+## Schema
+
+See [`docs/schema.md`](docs/schema.md) for the full session JSON schema with an example. The contract lives in one file: [`ams/schema.py`](ams/schema.py).
+
+## Frontend
+
+A simple frontend to browse and filter sessions is planned (not built yet). See [`docs/frontend-notes.md`](docs/frontend-notes.md) for the intended design — it reads the `index/` summaries to list sessions and fetches a full session JSON on click.
+
+## Development
+
+```bash
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[dev]"
+pytest
+```
+
+## License
+
+MIT

ams_observability-0.1.0/ams/__init__.py

@@ -0,0 +1,38 @@
+"""AMS — a super simple monitoring system for Claude agents.
+
+Capture a whole Claude Agent SDK session end to end — every tool call, every
+subagent and why it was invoked, the model's reasoning, results, timing and
+cost — as one readable JSON object in blob storage.
+
+    from ams.claude import traced_query
+
+    async for message in traced_query(prompt="...", options=options):
+        ...
+"""
+
+from .schema import (
+    SCHEMA_VERSION,
+    Agent,
+    Event,
+    EventType,
+    Session,
+    Status,
+    Totals,
+    Usage,
+)
+from .tracer import Tracer
+
+__version__ = "0.1.0"
+
+__all__ = [
+    "Tracer",
+    "Session",
+    "Event",
+    "EventType",
+    "Status",
+    "Totals",
+    "Usage",
+    "Agent",
+    "SCHEMA_VERSION",
+    "__version__",
+]

ams_observability-0.1.0/ams/claude.py

@@ -0,0 +1,46 @@
+"""Glue between AMS and `claude_agent_sdk`. This is the whole integration
+surface: swap `query` for `traced_query`, or merge `tracer.hooks()` into your
+options if you drive a ClaudeSDKClient yourself."""
+
+from __future__ import annotations
+
+from typing import Any, AsyncIterator, Optional
+
+from .tracer import Tracer
+
+
+def instrument_options(options: Any, tracer: Tracer) -> Any:
+    """Merge AMS hooks into an existing ClaudeAgentOptions, keeping any of yours."""
+    merged = dict(getattr(options, "hooks", None) or {})
+    for name, matchers in tracer.hooks().items():
+        merged[name] = list(merged.get(name, [])) + list(matchers)
+    options.hooks = merged
+    return options
+
+
+async def traced_query(
+    *,
+    prompt: Any,
+    options: Any = None,
+    tracer: Optional[Tracer] = None,
+    **tracer_kwargs: Any,
+) -> AsyncIterator[Any]:
+    """Drop-in replacement for `claude_agent_sdk.query` that records the session.
+
+        from ams.claude import traced_query
+
+        async for message in traced_query(prompt="...", options=options):
+            print(message)
+
+    Extra keyword args (storage, agent, environment, tags, metadata, redact)
+    are forwarded to `Tracer`. On stream completion the session is written to
+    storage automatically.
+    """
+    from claude_agent_sdk import ClaudeAgentOptions, query
+
+    tracer = tracer or Tracer(**tracer_kwargs)
+    options = options or ClaudeAgentOptions()
+    options = instrument_options(options, tracer)
+
+    async for message in tracer.watch(query(prompt=prompt, options=options)):
+        yield message

ams_observability-0.1.0/ams/pricing.py

@@ -0,0 +1,45 @@
+"""Token -> USD cost. The Claude Agent SDK already reports `total_cost_usd` on
+the result message, so AMS uses that for the session total. This table is a
+fallback for costing an individual LLM call from its token usage.
+
+Prices are USD per million tokens. Update as needed; matching is by substring so
+dated model ids (e.g. `claude-opus-4-8-20260101`) resolve to the right family.
+"""
+
+from __future__ import annotations
+
+from typing import Optional
+
+from .schema import Usage
+
+# (input, output, cache_write, cache_read) per million tokens
+_PRICES: dict[str, tuple[float, float, float, float]] = {
+    "claude-opus-4": (15.0, 75.0, 18.75, 1.5),
+    "claude-sonnet-4": (3.0, 15.0, 3.75, 0.3),
+    "claude-haiku-4": (1.0, 5.0, 1.25, 0.1),
+    "claude-3-5-haiku": (0.8, 4.0, 1.0, 0.08),
+}
+
+
+def _match(model: str) -> Optional[tuple[float, float, float, float]]:
+    model = model.lower()
+    for key, price in _PRICES.items():
+        if key in model:
+            return price
+    return None
+
+
+def cost_usd(model: Optional[str], usage: Optional[Usage]) -> Optional[float]:
+    if not model or usage is None:
+        return None
+    price = _match(model)
+    if price is None:
+        return None
+    p_in, p_out, p_cw, p_cr = price
+    total = (
+        usage.input_tokens * p_in
+        + usage.output_tokens * p_out
+        + usage.cache_creation_input_tokens * p_cw
+        + usage.cache_read_input_tokens * p_cr
+    )
+    return round(total / 1_000_000, 6)

ams_observability-0.1.0/ams/redact.py

@@ -0,0 +1,31 @@
+"""Optional PII redaction. Off by default — AMS captures full detail unless you
+opt in. Turn it on with `Tracer(redact=True)` or `AMS_REDACT=1` when sessions
+may contain sensitive caller data (phone numbers, emails, cards)."""
+
+from __future__ import annotations
+
+import re
+from typing import Any
+
+_PATTERNS = [
+    (re.compile(r"\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}\b"), "[email]"),
+    (re.compile(r"\+?\d[\d\s().-]{7,}\d"), "[phone]"),
+    (re.compile(r"\b(?:\d[ -]*?){13,16}\b"), "[card]"),
+    (re.compile(r"\b\d{3}-\d{2}-\d{4}\b"), "[ssn]"),
+]
+
+
+def redact_text(text: str) -> str:
+    for pattern, replacement in _PATTERNS:
+        text = pattern.sub(replacement, text)
+    return text
+
+
+def redact(value: Any) -> Any:
+    if isinstance(value, str):
+        return redact_text(value)
+    if isinstance(value, dict):
+        return {k: redact(v) for k, v in value.items()}
+    if isinstance(value, list):
+        return [redact(v) for v in value]
+    return value