agentlens-tracer 0.3.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- agentlens_tracer-0.3.0/.github/ISSUE_TEMPLATE/bug_report.md +40 -0
- agentlens_tracer-0.3.0/.github/ISSUE_TEMPLATE/config.yml +5 -0
- agentlens_tracer-0.3.0/.github/ISSUE_TEMPLATE/feature_request.md +25 -0
- agentlens_tracer-0.3.0/.github/workflows/ci.yml +129 -0
- agentlens_tracer-0.3.0/.github/workflows/demo.yml +48 -0
- agentlens_tracer-0.3.0/.github/workflows/release.yml +132 -0
- agentlens_tracer-0.3.0/.gitignore +12 -0
- agentlens_tracer-0.3.0/CHANGELOG.md +97 -0
- agentlens_tracer-0.3.0/CODE_OF_CONDUCT.md +56 -0
- agentlens_tracer-0.3.0/CONTRIBUTING.md +117 -0
- agentlens_tracer-0.3.0/LICENSE +21 -0
- agentlens_tracer-0.3.0/Makefile +45 -0
- agentlens_tracer-0.3.0/PKG-INFO +299 -0
- agentlens_tracer-0.3.0/README.md +254 -0
- agentlens_tracer-0.3.0/SECURITY.md +35 -0
- agentlens_tracer-0.3.0/agent_lens/__init__.py +81 -0
- agentlens_tracer-0.3.0/agent_lens/_textutil.py +80 -0
- agentlens_tracer-0.3.0/agent_lens/cli.py +298 -0
- agentlens_tracer-0.3.0/agent_lens/compare.py +118 -0
- agentlens_tracer-0.3.0/agent_lens/control.py +252 -0
- agentlens_tracer-0.3.0/agent_lens/dashboard/app.js +906 -0
- agentlens_tracer-0.3.0/agent_lens/dashboard/index.html +146 -0
- agentlens_tracer-0.3.0/agent_lens/dashboard/style.css +841 -0
- agentlens_tracer-0.3.0/agent_lens/dashboard_launcher.py +101 -0
- agentlens_tracer-0.3.0/agent_lens/export.py +217 -0
- agentlens_tracer-0.3.0/agent_lens/integrations/__init__.py +6 -0
- agentlens_tracer-0.3.0/agent_lens/integrations/_pricing.py +56 -0
- agentlens_tracer-0.3.0/agent_lens/integrations/anthropic.py +268 -0
- agentlens_tracer-0.3.0/agent_lens/integrations/langchain.py +332 -0
- agentlens_tracer-0.3.0/agent_lens/integrations/llamaindex.py +157 -0
- agentlens_tracer-0.3.0/agent_lens/integrations/openai.py +249 -0
- agentlens_tracer-0.3.0/agent_lens/mcp_server.py +176 -0
- agentlens_tracer-0.3.0/agent_lens/models.py +110 -0
- agentlens_tracer-0.3.0/agent_lens/server.py +509 -0
- agentlens_tracer-0.3.0/agent_lens/store.py +574 -0
- agentlens_tracer-0.3.0/agent_lens/tracer.py +438 -0
- agentlens_tracer-0.3.0/demo.gif +0 -0
- agentlens_tracer-0.3.0/demo.tape +16 -0
- agentlens_tracer-0.3.0/docs/ARCHITECTURE.md +313 -0
- agentlens_tracer-0.3.0/docs/FIRST_RUN.md +123 -0
- agentlens_tracer-0.3.0/docs/PROBLEM.md +103 -0
- agentlens_tracer-0.3.0/docs/SECURITY.md +86 -0
- agentlens_tracer-0.3.0/docs/api-reference.md +394 -0
- agentlens_tracer-0.3.0/docs/demo-storyboard.md +119 -0
- agentlens_tracer-0.3.0/docs/getting-started.md +165 -0
- agentlens_tracer-0.3.0/docs/pause-and-fork.md +213 -0
- agentlens_tracer-0.3.0/examples/01_openai_quickstart.py +0 -0
- agentlens_tracer-0.3.0/examples/02_anthropic_tools.py +200 -0
- agentlens_tracer-0.3.0/examples/03_langchain_chain.py +187 -0
- agentlens_tracer-0.3.0/examples/04_pause_and_fork.py +254 -0
- agentlens_tracer-0.3.0/examples/05_demo_recording.py +136 -0
- agentlens_tracer-0.3.0/examples/06_demo_cli.py +212 -0
- agentlens_tracer-0.3.0/examples/07_demo_mock.py +244 -0
- agentlens_tracer-0.3.0/examples/08_langchain_hypothesis.py +198 -0
- agentlens_tracer-0.3.0/examples/09_llamaindex_tracing.py +97 -0
- agentlens_tracer-0.3.0/notebooks/quickstart.ipynb +330 -0
- agentlens_tracer-0.3.0/pyproject.toml +97 -0
- agentlens_tracer-0.3.0/tests/__init__.py +1 -0
- agentlens_tracer-0.3.0/tests/conftest.py +52 -0
- agentlens_tracer-0.3.0/tests/integration/__init__.py +1 -0
- agentlens_tracer-0.3.0/tests/integration/test_concurrency.py +188 -0
- agentlens_tracer-0.3.0/tests/integration/test_e2e_anthropic.py +267 -0
- agentlens_tracer-0.3.0/tests/integration/test_e2e_openai.py +243 -0
- agentlens_tracer-0.3.0/tests/integration/test_overhead.py +151 -0
- agentlens_tracer-0.3.0/tests/integration/test_pause_fork.py +263 -0
- agentlens_tracer-0.3.0/tests/integrations/__init__.py +0 -0
- agentlens_tracer-0.3.0/tests/integrations/conftest.py +273 -0
- agentlens_tracer-0.3.0/tests/integrations/test_anthropic_integration.py +244 -0
- agentlens_tracer-0.3.0/tests/integrations/test_openai_integration.py +251 -0
- agentlens_tracer-0.3.0/tests/integrations/test_payload_helpers.py +113 -0
- agentlens_tracer-0.3.0/tests/integrations/test_pricing.py +76 -0
- agentlens_tracer-0.3.0/tests/integrations/test_sdk_surface.py +196 -0
- agentlens_tracer-0.3.0/tests/security/__init__.py +1 -0
- agentlens_tracer-0.3.0/tests/security/test_security.py +354 -0
- agentlens_tracer-0.3.0/tests/test_cli.py +241 -0
- agentlens_tracer-0.3.0/tests/test_compare.py +91 -0
- agentlens_tracer-0.3.0/tests/test_control.py +334 -0
- agentlens_tracer-0.3.0/tests/test_export.py +235 -0
- agentlens_tracer-0.3.0/tests/test_integrations.py +744 -0
- agentlens_tracer-0.3.0/tests/test_mcp_server.py +109 -0
- agentlens_tracer-0.3.0/tests/test_server.py +679 -0
- agentlens_tracer-0.3.0/tests/test_store_search.py +191 -0
- agentlens_tracer-0.3.0/tests/test_tracer.py +399 -0
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Bug report
|
|
3
|
+
about: Something broke — help us fix it
|
|
4
|
+
title: "[bug] "
|
|
5
|
+
labels: bug
|
|
6
|
+
assignees: ''
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## What happened
|
|
10
|
+
|
|
11
|
+
<!-- A clear description of the bug. -->
|
|
12
|
+
|
|
13
|
+
## Steps to reproduce
|
|
14
|
+
|
|
15
|
+
```python
|
|
16
|
+
# Minimal code to reproduce
|
|
17
|
+
```
|
|
18
|
+
|
|
19
|
+
1.
|
|
20
|
+
2.
|
|
21
|
+
3.
|
|
22
|
+
|
|
23
|
+
## Expected behavior
|
|
24
|
+
|
|
25
|
+
<!-- What should have happened? -->
|
|
26
|
+
|
|
27
|
+
## Actual behavior
|
|
28
|
+
|
|
29
|
+
<!-- What actually happened? Include the full traceback if applicable. -->
|
|
30
|
+
|
|
31
|
+
## Environment
|
|
32
|
+
|
|
33
|
+
- agent-lens version: <!-- pip show agentlens-tracer | grep Version -->
|
|
34
|
+
- Python version:
|
|
35
|
+
- OS:
|
|
36
|
+
- Framework being traced (OpenAI SDK / Anthropic SDK / LangChain / other):
|
|
37
|
+
|
|
38
|
+
## Anything else
|
|
39
|
+
|
|
40
|
+
<!-- Screenshots, related issues, workarounds you've tried. -->
|
|
@@ -0,0 +1,25 @@
|
|
|
1
|
+
---
|
|
2
|
+
name: Feature request
|
|
3
|
+
about: Propose a new capability
|
|
4
|
+
title: "[feat] "
|
|
5
|
+
labels: enhancement
|
|
6
|
+
assignees: ''
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Problem / use case
|
|
10
|
+
|
|
11
|
+
<!-- What are you trying to do that agent-lens doesn't support today? -->
|
|
12
|
+
<!-- "I want to X but currently I have to Y instead" is a great format. -->
|
|
13
|
+
|
|
14
|
+
## Proposed solution
|
|
15
|
+
|
|
16
|
+
<!-- What would you like to see? Be as concrete as possible. -->
|
|
17
|
+
<!-- API shape, CLI flag, dashboard button — whatever is relevant. -->
|
|
18
|
+
|
|
19
|
+
## Alternatives you've considered
|
|
20
|
+
|
|
21
|
+
<!-- Other tools, workarounds, or approaches you've tried. -->
|
|
22
|
+
|
|
23
|
+
## Additional context
|
|
24
|
+
|
|
25
|
+
<!-- Links, screenshots, prior art in other tools. -->
|
|
@@ -0,0 +1,129 @@
|
|
|
1
|
+
name: CI
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main, develop]
|
|
6
|
+
pull_request:
|
|
7
|
+
branches: [main]
|
|
8
|
+
|
|
9
|
+
jobs:
|
|
10
|
+
test:
|
|
11
|
+
name: "Python ${{ matrix.python-version }} on ${{ matrix.os }}"
|
|
12
|
+
runs-on: ${{ matrix.os }}
|
|
13
|
+
timeout-minutes: 15
|
|
14
|
+
strategy:
|
|
15
|
+
fail-fast: false
|
|
16
|
+
matrix:
|
|
17
|
+
os: [ubuntu-latest, macos-latest, windows-latest]
|
|
18
|
+
python-version: ["3.10", "3.11", "3.12"]
|
|
19
|
+
|
|
20
|
+
steps:
|
|
21
|
+
- uses: actions/checkout@v4
|
|
22
|
+
|
|
23
|
+
- name: Set up Python ${{ matrix.python-version }}
|
|
24
|
+
uses: actions/setup-python@v5
|
|
25
|
+
with:
|
|
26
|
+
python-version: ${{ matrix.python-version }}
|
|
27
|
+
cache: "pip"
|
|
28
|
+
|
|
29
|
+
- name: Install dependencies
|
|
30
|
+
run: pip install -e ".[dev]"
|
|
31
|
+
|
|
32
|
+
- name: Lint with ruff
|
|
33
|
+
run: ruff check agent_lens/ tests/ --output-format=github
|
|
34
|
+
|
|
35
|
+
- name: Run tests with coverage
|
|
36
|
+
shell: bash
|
|
37
|
+
run: |
|
|
38
|
+
pytest tests/ \
|
|
39
|
+
--cov=agent_lens \
|
|
40
|
+
--cov-report=xml \
|
|
41
|
+
--cov-report=term-missing \
|
|
42
|
+
-v
|
|
43
|
+
|
|
44
|
+
- name: Upload coverage to Codecov
|
|
45
|
+
uses: codecov/codecov-action@v5
|
|
46
|
+
if: matrix.os == 'ubuntu-latest' && matrix.python-version == '3.12'
|
|
47
|
+
with:
|
|
48
|
+
token: ${{ secrets.CODECOV_TOKEN }}
|
|
49
|
+
files: ./coverage.xml
|
|
50
|
+
flags: unittests
|
|
51
|
+
name: codecov-umbrella
|
|
52
|
+
fail_ci_if_error: false
|
|
53
|
+
|
|
54
|
+
sdk-surface:
|
|
55
|
+
name: Vendor SDK surface
|
|
56
|
+
runs-on: ubuntu-latest
|
|
57
|
+
timeout-minutes: 10
|
|
58
|
+
steps:
|
|
59
|
+
- uses: actions/checkout@v4
|
|
60
|
+
|
|
61
|
+
- name: Set up Python 3.12
|
|
62
|
+
uses: actions/setup-python@v5
|
|
63
|
+
with:
|
|
64
|
+
python-version: "3.12"
|
|
65
|
+
cache: "pip"
|
|
66
|
+
|
|
67
|
+
# The integration suite runs against stubs everywhere else, which proves
|
|
68
|
+
# the wrapper logic but not that it is bolted to attributes the vendors
|
|
69
|
+
# still ship. This job installs the real SDKs so that check actually runs
|
|
70
|
+
# instead of skipping.
|
|
71
|
+
- name: Install dependencies with vendor SDKs
|
|
72
|
+
run: pip install -e ".[dev,sdks]"
|
|
73
|
+
|
|
74
|
+
- name: Run integration tests against the real SDKs
|
|
75
|
+
run: pytest tests/integrations/ -v
|
|
76
|
+
|
|
77
|
+
security:
|
|
78
|
+
name: Security audit
|
|
79
|
+
runs-on: ubuntu-latest
|
|
80
|
+
timeout-minutes: 10
|
|
81
|
+
steps:
|
|
82
|
+
- uses: actions/checkout@v4
|
|
83
|
+
|
|
84
|
+
- name: Set up Python 3.12
|
|
85
|
+
uses: actions/setup-python@v5
|
|
86
|
+
with:
|
|
87
|
+
python-version: "3.12"
|
|
88
|
+
cache: "pip"
|
|
89
|
+
|
|
90
|
+
- name: Install pip-audit
|
|
91
|
+
run: pip install pip-audit
|
|
92
|
+
|
|
93
|
+
- name: Install project
|
|
94
|
+
run: pip install -e ".[dev]"
|
|
95
|
+
|
|
96
|
+
- name: Run pip-audit (project deps only)
|
|
97
|
+
# Audit project dependencies only; pip itself is excluded via --ignore-vuln
|
|
98
|
+
# CVE-2025-8869, CVE-2026-1703, CVE-2026-3219 are in pip, not our deps
|
|
99
|
+
run: |
|
|
100
|
+
pip-audit --skip-editable --desc \
|
|
101
|
+
--ignore-vuln CVE-2025-8869 \
|
|
102
|
+
--ignore-vuln CVE-2026-1703 \
|
|
103
|
+
--ignore-vuln CVE-2026-3219
|
|
104
|
+
continue-on-error: true
|
|
105
|
+
|
|
106
|
+
build:
|
|
107
|
+
name: Build distribution
|
|
108
|
+
runs-on: ubuntu-latest
|
|
109
|
+
timeout-minutes: 10
|
|
110
|
+
steps:
|
|
111
|
+
- uses: actions/checkout@v4
|
|
112
|
+
|
|
113
|
+
- name: Set up Python 3.12
|
|
114
|
+
uses: actions/setup-python@v5
|
|
115
|
+
with:
|
|
116
|
+
python-version: "3.12"
|
|
117
|
+
cache: "pip"
|
|
118
|
+
|
|
119
|
+
- name: Install build tools
|
|
120
|
+
run: pip install build hatchling
|
|
121
|
+
|
|
122
|
+
- name: Build package
|
|
123
|
+
run: python -m build
|
|
124
|
+
|
|
125
|
+
- name: Verify package contents
|
|
126
|
+
shell: bash
|
|
127
|
+
run: |
|
|
128
|
+
pip install dist/*.whl
|
|
129
|
+
python -c "import agent_lens; print(agent_lens.__version__)"
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
name: Generate Demo GIF
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
workflow_dispatch:
|
|
5
|
+
push:
|
|
6
|
+
branches: [main]
|
|
7
|
+
paths:
|
|
8
|
+
- demo.tape
|
|
9
|
+
- examples/07_demo_mock.py
|
|
10
|
+
|
|
11
|
+
permissions:
|
|
12
|
+
contents: write
|
|
13
|
+
|
|
14
|
+
jobs:
|
|
15
|
+
generate:
|
|
16
|
+
name: Render demo.gif with VHS
|
|
17
|
+
runs-on: ubuntu-latest
|
|
18
|
+
|
|
19
|
+
steps:
|
|
20
|
+
- uses: actions/checkout@v4
|
|
21
|
+
|
|
22
|
+
- name: Set up Python 3.12
|
|
23
|
+
uses: actions/setup-python@v5
|
|
24
|
+
with:
|
|
25
|
+
python-version: "3.12"
|
|
26
|
+
cache: pip
|
|
27
|
+
|
|
28
|
+
- name: Install system dependencies
|
|
29
|
+
run: |
|
|
30
|
+
sudo apt-get update -qq
|
|
31
|
+
sudo apt-get install -y ffmpeg ttyd
|
|
32
|
+
|
|
33
|
+
- name: Install agent-lens
|
|
34
|
+
run: pip install -e ".[dev]"
|
|
35
|
+
|
|
36
|
+
- name: Render demo.gif
|
|
37
|
+
uses: charmbracelet/vhs-action@v2
|
|
38
|
+
with:
|
|
39
|
+
path: demo.tape
|
|
40
|
+
install-dependencies: false
|
|
41
|
+
|
|
42
|
+
- name: Commit and push demo.gif
|
|
43
|
+
run: |
|
|
44
|
+
git config user.name "github-actions[bot]"
|
|
45
|
+
git config user.email "41898282+github-actions[bot]@users.noreply.github.com"
|
|
46
|
+
git add demo.gif
|
|
47
|
+
git diff --staged --quiet && echo "demo.gif unchanged, skipping commit" || \
|
|
48
|
+
git commit -m "chore: regenerate demo.gif [skip ci]" && git push
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
name: Release
|
|
2
|
+
|
|
3
|
+
# Publishes to PyPI when a version tag is pushed:
|
|
4
|
+
#
|
|
5
|
+
# git tag -a v0.3.0 -m "0.3.0" && git push origin v0.3.0
|
|
6
|
+
#
|
|
7
|
+
# Authentication is PyPI Trusted Publishing (OIDC). There is deliberately no API
|
|
8
|
+
# token anywhere in this repo: GitHub mints a short-lived credential for this
|
|
9
|
+
# workflow, PyPI verifies it came from this repo/workflow/environment, and
|
|
10
|
+
# nothing long-lived exists to leak, rotate, or accidentally commit.
|
|
11
|
+
#
|
|
12
|
+
# One-time setup on PyPI, at
|
|
13
|
+
# https://pypi.org/manage/project/agentlens-tracer/settings/publishing/ — the
|
|
14
|
+
# four values must match this file exactly:
|
|
15
|
+
#
|
|
16
|
+
# Owner: RAJUSHANIGARAPU
|
|
17
|
+
# Repository name: agent-lens
|
|
18
|
+
# Workflow name: release.yml
|
|
19
|
+
# Environment name: pypi
|
|
20
|
+
|
|
21
|
+
on:
|
|
22
|
+
push:
|
|
23
|
+
tags:
|
|
24
|
+
- "v*"
|
|
25
|
+
workflow_dispatch:
|
|
26
|
+
inputs:
|
|
27
|
+
target:
|
|
28
|
+
description: "Where to publish"
|
|
29
|
+
required: true
|
|
30
|
+
default: testpypi
|
|
31
|
+
type: choice
|
|
32
|
+
options:
|
|
33
|
+
- testpypi
|
|
34
|
+
- pypi
|
|
35
|
+
|
|
36
|
+
permissions:
|
|
37
|
+
contents: read
|
|
38
|
+
|
|
39
|
+
jobs:
|
|
40
|
+
# A tag is not a promise that the code is good — re-run the suite against the
|
|
41
|
+
# exact commit being published rather than trusting that CI passed on a branch.
|
|
42
|
+
verify:
|
|
43
|
+
name: Verify before publishing
|
|
44
|
+
runs-on: ubuntu-latest
|
|
45
|
+
timeout-minutes: 15
|
|
46
|
+
steps:
|
|
47
|
+
- uses: actions/checkout@v4
|
|
48
|
+
|
|
49
|
+
- name: Set up Python 3.12
|
|
50
|
+
uses: actions/setup-python@v5
|
|
51
|
+
with:
|
|
52
|
+
python-version: "3.12"
|
|
53
|
+
cache: "pip"
|
|
54
|
+
|
|
55
|
+
- name: Install dependencies with vendor SDKs
|
|
56
|
+
run: pip install -e ".[dev,sdks]"
|
|
57
|
+
|
|
58
|
+
- name: Lint
|
|
59
|
+
run: ruff check agent_lens/ tests/
|
|
60
|
+
|
|
61
|
+
- name: Test, including the vendor surface checks
|
|
62
|
+
run: pytest tests/ -q
|
|
63
|
+
|
|
64
|
+
build:
|
|
65
|
+
name: Build distribution
|
|
66
|
+
needs: verify
|
|
67
|
+
runs-on: ubuntu-latest
|
|
68
|
+
timeout-minutes: 10
|
|
69
|
+
steps:
|
|
70
|
+
- uses: actions/checkout@v4
|
|
71
|
+
|
|
72
|
+
- name: Set up Python 3.12
|
|
73
|
+
uses: actions/setup-python@v5
|
|
74
|
+
with:
|
|
75
|
+
python-version: "3.12"
|
|
76
|
+
cache: "pip"
|
|
77
|
+
|
|
78
|
+
- name: Install build tools
|
|
79
|
+
run: pip install build twine
|
|
80
|
+
|
|
81
|
+
- name: Build
|
|
82
|
+
run: python -m build
|
|
83
|
+
|
|
84
|
+
# A tag of v1.2.3 must not ship a package built from version 1.2.2. Without
|
|
85
|
+
# this the mismatch is only visible after it is on PyPI and unfixable.
|
|
86
|
+
- name: Check the tag matches the packaged version
|
|
87
|
+
if: startsWith(github.ref, 'refs/tags/')
|
|
88
|
+
shell: bash
|
|
89
|
+
run: |
|
|
90
|
+
tag="${GITHUB_REF_NAME#v}"
|
|
91
|
+
built="$(ls dist/*.tar.gz | sed -E 's/.*-([0-9][^-]*)\.tar\.gz/\1/')"
|
|
92
|
+
echo "tag=$tag packaged=$built"
|
|
93
|
+
if [ "$tag" != "$built" ]; then
|
|
94
|
+
echo "::error::tag $GITHUB_REF_NAME does not match packaged version $built"
|
|
95
|
+
exit 1
|
|
96
|
+
fi
|
|
97
|
+
|
|
98
|
+
- name: Validate metadata
|
|
99
|
+
run: twine check dist/*
|
|
100
|
+
|
|
101
|
+
- uses: actions/upload-artifact@v4
|
|
102
|
+
with:
|
|
103
|
+
name: dist
|
|
104
|
+
path: dist/
|
|
105
|
+
|
|
106
|
+
publish:
|
|
107
|
+
name: Publish to PyPI
|
|
108
|
+
needs: build
|
|
109
|
+
runs-on: ubuntu-latest
|
|
110
|
+
timeout-minutes: 10
|
|
111
|
+
# Named environment is half of what PyPI verifies in the OIDC claim; it also
|
|
112
|
+
# gives you a place to add a manual approval gate later if you want one.
|
|
113
|
+
environment:
|
|
114
|
+
name: ${{ github.event.inputs.target || 'pypi' }}
|
|
115
|
+
url: https://pypi.org/project/agentlens-tracer/
|
|
116
|
+
permissions:
|
|
117
|
+
id-token: write # mint the OIDC token; this is the whole credential story
|
|
118
|
+
steps:
|
|
119
|
+
- uses: actions/download-artifact@v4
|
|
120
|
+
with:
|
|
121
|
+
name: dist
|
|
122
|
+
path: dist/
|
|
123
|
+
|
|
124
|
+
- name: Publish to TestPyPI
|
|
125
|
+
if: github.event.inputs.target == 'testpypi'
|
|
126
|
+
uses: pypa/gh-action-pypi-publish@release/v1
|
|
127
|
+
with:
|
|
128
|
+
repository-url: https://test.pypi.org/legacy/
|
|
129
|
+
|
|
130
|
+
- name: Publish to PyPI
|
|
131
|
+
if: github.event.inputs.target != 'testpypi'
|
|
132
|
+
uses: pypa/gh-action-pypi-publish@release/v1
|
|
@@ -0,0 +1,97 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to agent-lens are documented here.
|
|
4
|
+
|
|
5
|
+
## [0.3.0] - 2026-09-02
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
- `agent_lens.install()` raised `AttributeError` on any environment with the
|
|
9
|
+
OpenAI SDK installed. The integration patched `Completions.acreate`, which no
|
|
10
|
+
openai >= 1.0 has ever defined, so the documented one-line entry point failed
|
|
11
|
+
for the SDK it is primarily meant to trace. Affects 0.1.0 and 0.2.0.
|
|
12
|
+
- Async LLM calls were not captured for either provider. The async call sites
|
|
13
|
+
are `AsyncCompletions.create` and `AsyncMessages.create`; both integrations
|
|
14
|
+
looked for an `acreate` attribute that no shipped SDK exposes, so the async
|
|
15
|
+
branches were unreachable and async calls were traced by nothing, silently.
|
|
16
|
+
- `gpt-4o-mini` calls were billed at `gpt-4o` rates. Pricing families are matched
|
|
17
|
+
as substrings and `gpt-4o` is contained in `gpt-4o-mini`, so a first-match scan
|
|
18
|
+
over-reported cost by 25–33x depending on the input/output token mix. Matching
|
|
19
|
+
now takes the longest key, which also fixes `gpt-4-turbo` and dated snapshots
|
|
20
|
+
such as `claude-3-5-sonnet-20241022`.
|
|
21
|
+
- `unpatch()` never restored the original SDK callables — it only cleared the
|
|
22
|
+
patched flag, so a patch/unpatch/patch cycle wrapped the wrapper and emitted
|
|
23
|
+
duplicate spans and events for every call.
|
|
24
|
+
- An Anthropic message object that raised during serialisation propagated out of
|
|
25
|
+
the traced call. Both integrations now degrade to a described placeholder
|
|
26
|
+
rather than breaking the application being observed.
|
|
27
|
+
- `test_llamaindex_noop_when_unavailable` inferred "llama-index is not installed"
|
|
28
|
+
from the environment, so it inverted and failed in any job that does install
|
|
29
|
+
it. It now forces the unavailable path explicitly.
|
|
30
|
+
- The `tests/integration` end-to-end suite believed it was mocked and was making
|
|
31
|
+
**real calls to api.openai.com and api.anthropic.com**. respx no longer
|
|
32
|
+
intercepts these SDKs — verified against openai 3.7.0 / anthropic 1.3.0 /
|
|
33
|
+
httpx 0.28.1 / respx 0.23.1, where the route is never matched and the request
|
|
34
|
+
reaches the vendor. It went unnoticed because the suite skipped whenever the
|
|
35
|
+
provider SDK was absent, which the `dev` extra guarantees. Run with a real key
|
|
36
|
+
exported it would have billed the account and sent the prompts upstream. The
|
|
37
|
+
suite is now gated behind an explicit `AGENT_LENS_LIVE_E2E=1` opt-in, so a live
|
|
38
|
+
call is a decision rather than a side effect of installing a package.
|
|
39
|
+
|
|
40
|
+
### Added
|
|
41
|
+
- `tests/integrations/test_sdk_surface.py` — asserts that the attributes the
|
|
42
|
+
integrations patch actually exist on the installed vendor SDKs, and that
|
|
43
|
+
`install()` returns rather than raises. Coverage did not catch any of the bugs
|
|
44
|
+
above: the capture layer was at 85–93% and four tests specifically exercised
|
|
45
|
+
the async path, all passing, because the fake SDK was shaped around what the
|
|
46
|
+
code called instead of around what the vendors ship. Every other test in the
|
|
47
|
+
suite runs against stubs by design — this is the one that checks the seam.
|
|
48
|
+
- Surface checks for the two callback integrations as well. They have the same
|
|
49
|
+
seam in a different shape: they subclass a vendor base and override the hooks
|
|
50
|
+
the framework calls, so a renamed hook would simply never fire — no error, no
|
|
51
|
+
span, no trace. The check asserts that the real base class was imported (not
|
|
52
|
+
the `object` fallback, which would swallow a rename silently), that every hook
|
|
53
|
+
the handler overrides exists upstream, and that the handler instantiates
|
|
54
|
+
against the real base. The hook list is derived by introspection, so a hook
|
|
55
|
+
added later is checked automatically. Verified against langchain-core 1.6.1 and
|
|
56
|
+
llama-index-core 0.14.24: both are currently correct.
|
|
57
|
+
- A `sdks` extra and a matching `sdk-surface` CI job that installs the real
|
|
58
|
+
openai, anthropic, langchain-core and llama-index-core packages, so the surface
|
|
59
|
+
checks run instead of skipping. The default matrix stays offline and fast.
|
|
60
|
+
- Direct unit coverage for the payload builders and pricing resolution, taking
|
|
61
|
+
`openai.py` and `anthropic.py` to 100%.
|
|
62
|
+
|
|
63
|
+
### Changed
|
|
64
|
+
- Async spans are named `openai.achat(...)` and `anthropic.amessages(...)`, and
|
|
65
|
+
now record the same payload as their sync counterparts — the async paths
|
|
66
|
+
previously dropped `kwargs`, the Anthropic system prompt, and thinking blocks.
|
|
67
|
+
|
|
68
|
+
## [0.2.0] - 2026-05-18
|
|
69
|
+
|
|
70
|
+
### Added
|
|
71
|
+
- LlamaIndex callback handler (`AgentLensLlamaIndexHandler`) — trace any LlamaIndex query engine or chat engine with zero code changes
|
|
72
|
+
- GitHub issue templates for bug reports and feature requests
|
|
73
|
+
- Colab quickstart notebook — full hypothesis → fork → diff workflow in 5 minutes, no API key required
|
|
74
|
+
- LangChain hypothesis testing example (`examples/08_langchain_hypothesis.py`)
|
|
75
|
+
- Automated demo GIF generation via GitHub Actions (VHS)
|
|
76
|
+
- Good-first-issues table in CONTRIBUTING.md
|
|
77
|
+
|
|
78
|
+
### Fixed
|
|
79
|
+
- VHS CI action: pre-install ffmpeg and ttyd to avoid install failures on ubuntu-latest
|
|
80
|
+
|
|
81
|
+
## [0.1.0] - 2026-05-02
|
|
82
|
+
|
|
83
|
+
### Added
|
|
84
|
+
- Core tracer with OpenAI SDK auto-patch (`agent_lens.install()`)
|
|
85
|
+
- Anthropic SDK integration with extended thinking capture
|
|
86
|
+
- LangChain callback handler (`AgentLensCallbackHandler`)
|
|
87
|
+
- FastAPI dashboard server at `localhost:7878`
|
|
88
|
+
- SQLite store — all traces persist in `~/.agent-lens/runs.db`
|
|
89
|
+
- `POST /runs/{id}/fork` — fork with hypothesis notes and expected_output assertion
|
|
90
|
+
- `GET /runs/{a}/diff/{b}` — structural diff: messages, metrics delta, verdict
|
|
91
|
+
- `GET /runs/{id}/lineage` — full ancestry chain of forked runs
|
|
92
|
+
- `POST /runs/{id}/note` — annotate any run after the fact
|
|
93
|
+
- Pause / resume control plane — blocks agent at next LLM call
|
|
94
|
+
- Secret redaction — Bearer tokens, `sk-*`, `AIza*`, `sk-ant-*` stripped before SQLite
|
|
95
|
+
- Self-contained HTML export — `agent-lens export <run_id> --output trace.html`
|
|
96
|
+
- Real-time SSE dashboard with span tree, flame graph, message inspector
|
|
97
|
+
- `@agent_lens.trace` decorator for any Python function
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# Contributor Covenant Code of Conduct
|
|
2
|
+
|
|
3
|
+
## Our Pledge
|
|
4
|
+
|
|
5
|
+
We as members, contributors, and leaders pledge to make participation in our
|
|
6
|
+
community a harassment-free experience for everyone, regardless of age, body
|
|
7
|
+
size, visible or invisible disability, ethnicity, sex characteristics, gender
|
|
8
|
+
identity and expression, level of experience, education, socio-economic status,
|
|
9
|
+
nationality, personal appearance, race, caste, color, religion, or sexual
|
|
10
|
+
identity and orientation.
|
|
11
|
+
|
|
12
|
+
We pledge to act and interact in ways that contribute to an open, welcoming,
|
|
13
|
+
diverse, inclusive, and healthy community.
|
|
14
|
+
|
|
15
|
+
## Our Standards
|
|
16
|
+
|
|
17
|
+
Examples of behavior that contributes to a positive environment:
|
|
18
|
+
|
|
19
|
+
* Demonstrating empathy and kindness toward other people
|
|
20
|
+
* Being respectful of differing opinions, viewpoints, and experiences
|
|
21
|
+
* Giving and gracefully accepting constructive feedback
|
|
22
|
+
* Accepting responsibility and apologizing to those affected by our mistakes
|
|
23
|
+
* Focusing on what is best not just for us as individuals, but for the overall community
|
|
24
|
+
|
|
25
|
+
Examples of unacceptable behavior:
|
|
26
|
+
|
|
27
|
+
* The use of sexualized language or imagery, and sexual attention or advances of any kind
|
|
28
|
+
* Trolling, insulting or derogatory comments, and personal or political attacks
|
|
29
|
+
* Public or private harassment
|
|
30
|
+
* Publishing others' private information without their explicit permission
|
|
31
|
+
* Other conduct which could reasonably be considered inappropriate in a professional setting
|
|
32
|
+
|
|
33
|
+
## Enforcement Responsibilities
|
|
34
|
+
|
|
35
|
+
Project maintainers are responsible for clarifying and enforcing our standards and
|
|
36
|
+
will take appropriate and fair corrective action in response to any behavior that
|
|
37
|
+
they deem inappropriate, threatening, offensive, or harmful.
|
|
38
|
+
|
|
39
|
+
## Scope
|
|
40
|
+
|
|
41
|
+
This Code of Conduct applies within all community spaces, and also applies when
|
|
42
|
+
an individual is officially representing the community in public spaces.
|
|
43
|
+
|
|
44
|
+
## Enforcement
|
|
45
|
+
|
|
46
|
+
Instances of abusive, harassing, or otherwise unacceptable behavior may be
|
|
47
|
+
reported to the project maintainer at rajub4u927@gmail.com.
|
|
48
|
+
All complaints will be reviewed and investigated promptly and fairly.
|
|
49
|
+
|
|
50
|
+
## Attribution
|
|
51
|
+
|
|
52
|
+
This Code of Conduct is adapted from the [Contributor Covenant][homepage],
|
|
53
|
+
version 2.1, available at
|
|
54
|
+
https://www.contributor-covenant.org/version/2/1/code_of_conduct.html.
|
|
55
|
+
|
|
56
|
+
[homepage]: https://www.contributor-covenant.org
|
|
@@ -0,0 +1,117 @@
|
|
|
1
|
+
# Contributing to agent-lens
|
|
2
|
+
|
|
3
|
+
Thank you for your interest in contributing! agent-lens is an early-stage open-source project and all contributions are genuinely welcome.
|
|
4
|
+
|
|
5
|
+
## Ways to Contribute
|
|
6
|
+
|
|
7
|
+
- **Bug reports** — Open an issue with steps to reproduce, expected behavior, actual behavior, and your Python version.
|
|
8
|
+
- **Feature requests** — Open an issue describing the use case (not just the feature). We prioritize based on "how many developers hit this problem."
|
|
9
|
+
- **Code** — See below.
|
|
10
|
+
- **Documentation** — Fix typos, add examples, clarify confusing sections.
|
|
11
|
+
- **Testing** — Add test cases, especially for edge cases and concurrency.
|
|
12
|
+
|
|
13
|
+
## Good First Issues
|
|
14
|
+
|
|
15
|
+
If you're new to the project, look for issues tagged `good first issue`:
|
|
16
|
+
|
|
17
|
+
| Area | Task |
|
|
18
|
+
|------|------|
|
|
19
|
+
| **Integrations** | Add a `LlamaIndexCallbackHandler` — mirror `agent_lens/integrations/langchain.py` |
|
|
20
|
+
| **Integrations** | Add AutoGen / CrewAI support via `@agent_lens.trace` wrapper |
|
|
21
|
+
| **CLI** | Add `agent-lens runs` to list recent runs in the terminal |
|
|
22
|
+
| **CLI** | Add `agent-lens diff <run_a> <run_b>` shorthand for the diff endpoint |
|
|
23
|
+
| **CLI** | Add `--json` flag to `agent-lens export` for JSON output instead of HTML |
|
|
24
|
+
| **Dashboard** | Show `cost_usd` in the run list table |
|
|
25
|
+
| **Dashboard** | Add a "copy curl command" button next to each run |
|
|
26
|
+
| **Tests** | Add integration test for `GET /lineage` with a 3-deep fork chain |
|
|
27
|
+
| **Async** | Async-aware pause/resume using `asyncio.Event` |
|
|
28
|
+
|
|
29
|
+
## Development Setup
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
# Clone
|
|
33
|
+
git clone https://github.com/RAJUSHANIGARAPU/agent-lens.git
|
|
34
|
+
cd agent-lens
|
|
35
|
+
|
|
36
|
+
# Install in editable mode with dev deps
|
|
37
|
+
pip install -e ".[dev]"
|
|
38
|
+
|
|
39
|
+
# Verify tests pass
|
|
40
|
+
pytest tests/ -v
|
|
41
|
+
```
|
|
42
|
+
|
|
43
|
+
## Running the Demo
|
|
44
|
+
|
|
45
|
+
```bash
|
|
46
|
+
# No API key needed — uses pre-canned responses
|
|
47
|
+
python examples/07_demo_mock.py
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
## Code Style
|
|
51
|
+
|
|
52
|
+
- **Ruff** for linting: `ruff check agent_lens/ tests/`
|
|
53
|
+
- **Type hints** on all public functions
|
|
54
|
+
- **Docstrings** on all public classes and functions
|
|
55
|
+
- No `@Autowired` (this is Python :) — constructor injection only
|
|
56
|
+
- Constructor injection for dependencies (no globals in business logic except the singletons in `tracer.py`, `control.py`, `store.py`)
|
|
57
|
+
|
|
58
|
+
## Testing
|
|
59
|
+
|
|
60
|
+
All new code needs tests. The test structure:
|
|
61
|
+
|
|
62
|
+
```
|
|
63
|
+
tests/
|
|
64
|
+
test_tracer.py — unit tests for the tracer
|
|
65
|
+
test_server.py — FastAPI app tests
|
|
66
|
+
test_control.py — ControlPlane unit tests
|
|
67
|
+
integration/ — end-to-end tests
|
|
68
|
+
security/ — security-focused tests
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
Run the full test suite:
|
|
72
|
+
```bash
|
|
73
|
+
pytest tests/ --cov=agent_lens -v
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Run a specific file:
|
|
77
|
+
```bash
|
|
78
|
+
pytest tests/test_tracer.py -v
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
## Pull Request Process
|
|
82
|
+
|
|
83
|
+
1. Fork the repo and create a feature branch: `git checkout -b feat/my-feature`
|
|
84
|
+
2. Write tests first (TDD encouraged but not required)
|
|
85
|
+
3. Implement the feature
|
|
86
|
+
4. Ensure all tests pass: `pytest tests/`
|
|
87
|
+
5. Lint: `ruff check agent_lens/ tests/`
|
|
88
|
+
6. Open a PR with a clear description of what changed and why
|
|
89
|
+
|
|
90
|
+
## Commit Message Format
|
|
91
|
+
|
|
92
|
+
```
|
|
93
|
+
type(scope): description
|
|
94
|
+
|
|
95
|
+
Types: feat, fix, chore, refactor, test, docs
|
|
96
|
+
Scope: tracer, store, control, server, dashboard, cli, integrations
|
|
97
|
+
|
|
98
|
+
Examples:
|
|
99
|
+
feat(control): add async pause/resume using asyncio.Event
|
|
100
|
+
fix(tracer): prevent context leak between async tasks
|
|
101
|
+
test(security): add path traversal test for replay command
|
|
102
|
+
docs(api): add WebSocket events documentation
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
## Architecture
|
|
106
|
+
|
|
107
|
+
Before making significant changes, read [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md). Key design constraints:
|
|
108
|
+
|
|
109
|
+
- The tracer overhead must stay < 5ms per call
|
|
110
|
+
- SQLite is the only storage backend (no PostgreSQL, no Redis)
|
|
111
|
+
- The server must bind to 127.0.0.1 by default
|
|
112
|
+
- No data must ever leave the machine
|
|
113
|
+
- No `eval()`, `exec()`, or `pickle` in any data deserialization path
|
|
114
|
+
|
|
115
|
+
## Questions?
|
|
116
|
+
|
|
117
|
+
Open a GitHub Discussion or email rajub4u927@gmail.com.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Raju S
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|