agent-readable 0.2.0__tar.gz → 0.3.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (34) hide show
  1. agent_readable-0.2.0/.github/workflows/test.yml → agent_readable-0.3.0/.github/workflows/ci.yml +10 -5
  2. {agent_readable-0.2.0 → agent_readable-0.3.0}/.github/workflows/publish.yml +5 -5
  3. {agent_readable-0.2.0 → agent_readable-0.3.0}/.gitignore +3 -0
  4. agent_readable-0.3.0/CHANGELOG.md +142 -0
  5. agent_readable-0.3.0/PKG-INFO +97 -0
  6. agent_readable-0.3.0/README.md +70 -0
  7. agent_readable-0.3.0/docs/authoring.md +79 -0
  8. agent_readable-0.3.0/docs/benchmark.md +79 -0
  9. agent_readable-0.3.0/docs/examples.md +253 -0
  10. agent_readable-0.3.0/docs/faq.md +50 -0
  11. agent_readable-0.3.0/docs/getting-started.md +153 -0
  12. agent_readable-0.3.0/docs/why.md +49 -0
  13. {agent_readable-0.2.0 → agent_readable-0.3.0}/pyproject.toml +12 -1
  14. agent_readable-0.3.0/scripts/benchmark.py +243 -0
  15. agent_readable-0.3.0/src/agent_readable/__main__.py +77 -0
  16. {agent_readable-0.2.0 → agent_readable-0.3.0}/src/agent_readable/_model.py +130 -17
  17. {agent_readable-0.2.0 → agent_readable-0.3.0}/src/agent_readable/_protocol.py +37 -47
  18. {agent_readable-0.2.0 → agent_readable-0.3.0}/src/agent_readable/_render.py +12 -2
  19. {agent_readable-0.2.0 → agent_readable-0.3.0}/tests/test_cli.py +31 -23
  20. {agent_readable-0.2.0 → agent_readable-0.3.0}/tests/test_protocol.py +251 -27
  21. agent_readable-0.2.0/CHANGELOG.md +0 -73
  22. agent_readable-0.2.0/PKG-INFO +0 -653
  23. agent_readable-0.2.0/README.md +0 -626
  24. agent_readable-0.2.0/src/agent_readable/__main__.py +0 -59
  25. {agent_readable-0.2.0 → agent_readable-0.3.0}/LICENSE +0 -0
  26. {agent_readable-0.2.0 → agent_readable-0.3.0}/docs/agent_help_vs_help.gif +0 -0
  27. {agent_readable-0.2.0 → agent_readable-0.3.0}/examples/any_class.py +0 -0
  28. {agent_readable-0.2.0 → agent_readable-0.3.0}/examples/duck_type.py +0 -0
  29. {agent_readable-0.2.0 → agent_readable-0.3.0}/examples/modules_and_functions.py +0 -0
  30. {agent_readable-0.2.0 → agent_readable-0.3.0}/examples/sqlite_connection.py +0 -0
  31. {agent_readable-0.2.0 → agent_readable-0.3.0}/examples/temperature.py +0 -0
  32. {agent_readable-0.2.0 → agent_readable-0.3.0}/src/agent_readable/__init__.py +0 -0
  33. {agent_readable-0.2.0 → agent_readable-0.3.0}/src/agent_readable/py.typed +0 -0
  34. {agent_readable-0.2.0 → agent_readable-0.3.0}/tests/__init__.py +0 -0
@@ -1,4 +1,4 @@
1
- name: Tests
1
+ name: CI
2
2
 
3
3
  on:
4
4
  push:
@@ -10,10 +10,10 @@ jobs:
10
10
  lint:
11
11
  runs-on: ubuntu-latest
12
12
  steps:
13
- - uses: actions/checkout@v4
13
+ - uses: actions/checkout@v6
14
14
 
15
15
  - name: Install uv
16
- uses: astral-sh/setup-uv@v3
16
+ uses: astral-sh/setup-uv@v8.2.0
17
17
 
18
18
  - name: Install dev deps
19
19
  run: uv sync --group dev
@@ -24,6 +24,9 @@ jobs:
24
24
  - name: Ruff format check
25
25
  run: uv run ruff format --check
26
26
 
27
+ - name: Mypy
28
+ run: uv run mypy
29
+
27
30
  test:
28
31
  runs-on: ubuntu-latest
29
32
  strategy:
@@ -32,10 +35,12 @@ jobs:
32
35
  python-version: ["3.10", "3.11", "3.12", "3.13", "3.14"]
33
36
 
34
37
  steps:
35
- - uses: actions/checkout@v4
38
+ - uses: actions/checkout@v6
36
39
 
37
40
  - name: Install uv
38
- uses: astral-sh/setup-uv@v3
41
+ uses: astral-sh/setup-uv@v8.2.0
42
+ with:
43
+ cache-suffix: py${{ matrix.python-version }}
39
44
 
40
45
  - name: Set up Python ${{ matrix.python-version }}
41
46
  run: uv python install ${{ matrix.python-version }}
@@ -9,16 +9,16 @@ jobs:
9
9
  build:
10
10
  runs-on: ubuntu-latest
11
11
  steps:
12
- - uses: actions/checkout@v4
12
+ - uses: actions/checkout@v6
13
13
 
14
14
  - name: Install uv
15
- uses: astral-sh/setup-uv@v3
15
+ uses: astral-sh/setup-uv@v8.2.0
16
16
 
17
17
  - name: Build sdist + wheel
18
18
  run: uv build
19
19
 
20
20
  - name: Upload artifacts
21
- uses: actions/upload-artifact@v4
21
+ uses: actions/upload-artifact@v7
22
22
  with:
23
23
  name: dist
24
24
  path: dist/
@@ -33,7 +33,7 @@ jobs:
33
33
  id-token: write
34
34
  steps:
35
35
  - name: Download artifacts
36
- uses: actions/download-artifact@v4
36
+ uses: actions/download-artifact@v8
37
37
  with:
38
38
  name: dist
39
39
  path: dist/
@@ -48,7 +48,7 @@ jobs:
48
48
  contents: write
49
49
  steps:
50
50
  - name: Download artifacts
51
- uses: actions/download-artifact@v4
51
+ uses: actions/download-artifact@v8
52
52
  with:
53
53
  name: dist
54
54
  path: dist/
@@ -43,3 +43,6 @@ hatch.toml
43
43
 
44
44
  # No external dependencies for this project, and uv.lock is only needed for local dev.
45
45
  uv.lock
46
+
47
+ # Local scratch files (audits, drafts, planning docs)
48
+ .localonly/
@@ -0,0 +1,142 @@
1
+ # Changelog
2
+
3
+ All notable changes to this project are documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
6
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
8
+ ## [Unreleased]
9
+
10
+ ## [0.3.0] - 2026-08-23
11
+
12
+ ### Added
13
+
14
+ - Console script `agent-readable`, so one-off use is
15
+ `uvx agent-readable sqlite3:Connection` instead of
16
+ `uvx --from agent-readable python -m agent_readable sqlite3:Connection`.
17
+ `python -m agent_readable` keeps working.
18
+ - `--version` flag for the CLI.
19
+ - Public class attributes and module constants are now listed in the Public API
20
+ section with their current value: `repr` for exact primitive types (never
21
+ executing a custom `__repr__`), the type name otherwise.
22
+ - Enum members are listed (e.g. ``- `RED` member: 1``), and the misleading
23
+ `EnumMeta.__call__` constructor signature is no longer shown for enum
24
+ classes.
25
+ - Module docs now include C builtins (`inspect.isroutine` instead of
26
+ `inspect.isfunction`), so `agent_help(math)` lists `sin` alongside
27
+ pure-Python functions, and module-level constants such as `math.pi` are no
28
+ longer dropped by the origin heuristic.
29
+ - A constructor that introspection renders as the placeholder `(*args,
30
+ **kwargs)` — e.g. masked by a metaclass `__call__` — now falls back to the
31
+ class's `__init__`/`__new__` signature.
32
+ - Mypy type-checking job in CI.
33
+ - Benchmark harness (`scripts/benchmark.py`) and methodology
34
+ (`docs/benchmark.md`) for measuring whether `agent_help()` context reduces
35
+ hallucinated-API failures versus a baseline prompt. Status: not yet run;
36
+ no numbers are claimed until a real run is recorded.
37
+
38
+ ### Changed
39
+
40
+ - **Breaking:** `__agent_notes__()` sections are now always appended, including
41
+ after a custom `__agent_help__()`'s output. Previously the notes were
42
+ silently dropped when a custom `__agent_help__` was defined and a
43
+ `UserWarning` was emitted; defining both is now a supported combination and
44
+ the warning is gone. For full verbatim control of the entire output, do not
45
+ define `__agent_notes__()` anywhere in the MRO.
46
+ - The CLI prints a one-line `error: ...` message to stderr and exits with
47
+ status 2 for targets that cannot be imported or resolved, instead of raising
48
+ a traceback.
49
+
50
+ ### Fixed
51
+
52
+ - A raising `__agent_notes__()` no longer breaks `agent_help()` for the class
53
+ and all of its subclasses; the broken notes are skipped, mirroring the
54
+ existing `__agent_help__()` fallback.
55
+ - `functools.cached_property` members are no longer silently dropped from the
56
+ Public API; they render as properties with the wrapped function's docstring.
57
+
58
+ ## [0.2.1] - 2026-07-04
59
+
60
+ ### Documentation
61
+
62
+ - Restructured the README into a concise overview and moved detailed guidance
63
+ into focused docs for getting started, examples, authoring, rationale, and
64
+ FAQ.
65
+ - Added CI and PyPI badges to the README.
66
+ - Added `uv add`, `uvx`, `uv tool run`, and `pipx run` installation and
67
+ one-off execution examples.
68
+ - Added the TypeScript implementation under "Other Languages".
69
+
70
+ ### Changed
71
+
72
+ - Renamed the GitHub Actions test workflow file to `ci.yml` and its display
73
+ name to `CI`.
74
+ - Updated GitHub Actions versions to Node 24-compatible releases.
75
+ - Made the uv cache key unique for each Python version in the CI matrix.
76
+
77
+ ## [0.1.2] - 2026-05-30
78
+
79
+ ### Added
80
+
81
+ - `py.typed` marker so inline type annotations are visible to downstream type
82
+ checkers (PEP 561).
83
+ - A module's `__all__` is now honored as the authoritative public API when
84
+ present, including symbols re-exported from other modules.
85
+ - `agent_help()` now emits a `UserWarning` when a class defines both a custom
86
+ `__agent_help__()` and `__agent_notes__()`, since the notes are silently
87
+ dropped in that case.
88
+ - Agent Skill at `skills/agent-readable/SKILL.md` following the
89
+ [Agent Skills open standard](https://agentskills.io). Installable via
90
+ `npx skills add zydo/agent-readable --skill agent-readable` and portable
91
+ across Claude Code, Codex CLI (OpenAI), Gemini CLI (Google), GitHub Copilot,
92
+ Cursor, JetBrains Junie, Goose, OpenCode, and 40+ other adopters. The skill
93
+ teaches the agent to call `agent_help()` before writing Python against a
94
+ library and to author new APIs with `__agent_notes__()`. Supersedes
95
+ `AGENT-PROMPT.md` as the recommended way to wire up coding agents.
96
+
97
+ ### Fixed
98
+
99
+ - Method signatures with a positional-only `self` no longer render a dangling
100
+ `/` (e.g. `backup(/, target)` now renders as `backup(target)`).
101
+
102
+ ### Documentation
103
+
104
+ - README and examples reframed around curation, not compactness. The "Why it
105
+ matters" section now leads with two failure modes — what-exists (curated
106
+ Public API list curbs hallucinated methods and stale signatures) and
107
+ how-to-use (lifecycle rules via `__agent_notes__()`) — instead of a
108
+ comparative empirical claim about which is more common. Removed the
109
+ "217 vs 56 lines" framing in favor of structure-and-rules language.
110
+ - Tagline sharpened to lead with the hallucination-stopping outcome rather
111
+ than the protocol mechanics.
112
+
113
+ ### Removed
114
+
115
+ - `AGENT-PROMPT.md` (superseded by the agent skill at
116
+ `skills/agent-readable/SKILL.md`).
117
+
118
+ ## [0.1.1] - 2026-05-11
119
+
120
+ ### Added
121
+
122
+ - `agent_help()` support for functions and methods.
123
+
124
+ ### Documentation
125
+
126
+ - Demo GIF comparing `agent_help()` and `help()`, plus additional README
127
+ examples.
128
+
129
+ ## [0.1.0] - 2026-05-10
130
+
131
+ ### Added
132
+
133
+ - Initial release: the `agent_help()` function, the `AgentReadable` protocol,
134
+ `AgentReadableMixin`, `__agent_notes__` accumulation across the MRO, module
135
+ support, and the `python -m agent_readable` CLI.
136
+
137
+ [Unreleased]: https://github.com/zydo/agent-readable/compare/v0.3.0...HEAD
138
+ [0.3.0]: https://github.com/zydo/agent-readable/compare/v0.2.1...v0.3.0
139
+ [0.2.1]: https://github.com/zydo/agent-readable/compare/v0.2.0...v0.2.1
140
+ [0.1.2]: https://github.com/zydo/agent-readable/compare/v0.1.1...v0.1.2
141
+ [0.1.1]: https://github.com/zydo/agent-readable/compare/v0.1.0...v0.1.1
142
+ [0.1.0]: https://github.com/zydo/agent-readable/releases/tag/v0.1.0
@@ -0,0 +1,97 @@
1
+ Metadata-Version: 2.5
2
+ Name: agent-readable
3
+ Version: 0.3.0
4
+ Summary: A lightweight Python protocol for agent-oriented documentation
5
+ Project-URL: Repository, https://github.com/zydo/agent-readable
6
+ Project-URL: Issues, https://github.com/zydo/agent-readable/issues
7
+ Project-URL: Changelog, https://github.com/zydo/agent-readable/blob/main/CHANGELOG.md
8
+ Author: zydo and agent-readable contributors
9
+ License-Expression: MIT
10
+ License-File: LICENSE
11
+ Keywords: agent,agent-help,ai,ai-agent,coding-agent,context-engineering,docstring,documentation,llm,mixin,prompt-engineering,protocol,vibe-coding
12
+ Classifier: Development Status :: 3 - Alpha
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Programming Language :: Python :: 3
15
+ Classifier: Programming Language :: Python :: 3.10
16
+ Classifier: Programming Language :: Python :: 3.11
17
+ Classifier: Programming Language :: Python :: 3.12
18
+ Classifier: Programming Language :: Python :: 3.13
19
+ Classifier: Programming Language :: Python :: 3.14
20
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
21
+ Classifier: Topic :: Software Development :: Code Generators
22
+ Classifier: Topic :: Software Development :: Documentation
23
+ Classifier: Topic :: Software Development :: Libraries :: Python Modules
24
+ Classifier: Typing :: Typed
25
+ Requires-Python: >=3.10
26
+ Description-Content-Type: text/markdown
27
+
28
+ # agent-readable
29
+
30
+ [![CI](https://github.com/zydo/agent-readable/actions/workflows/ci.yml/badge.svg)](https://github.com/zydo/agent-readable/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/agent-readable.svg)](https://pypi.org/project/agent-readable/)
31
+
32
+ Stop coding agents from hallucinating Python APIs. `agent_help(target)` reads the live public API surface from any Python class, module, function, or method and renders compact, agent-oriented Markdown with current signatures and docstring summaries. When a library opts in, it can also return class-level usage rules such as lifecycle order, preconditions, and anti-patterns.
33
+
34
+ To let your coding agent automatically call `agent_help()` before using an unfamiliar API, install the companion skill:
35
+
36
+ ```bash
37
+ npx skills add zydo/skills --skill agent-readable
38
+ ```
39
+
40
+ <!-- markdownlint-disable MD033 -->
41
+ <p align="center">
42
+ <strong><code>logging.Logger</code> compared with <code>agent_help()</code> and <code>help()</code></strong><br>
43
+ <img src="https://raw.githubusercontent.com/zydo/agent-readable/main/docs/agent_help_vs_help.gif" alt="agent_help vs help">
44
+ </p>
45
+ <!-- markdownlint-enable MD033 -->
46
+
47
+ ## Quickstart
48
+
49
+ See [Getting started](docs/getting-started.md) for installation, one-off CLI usage with `uvx`, `uv tool run`, or `pipx`, and full CLI examples.
50
+
51
+ ```python
52
+ from agent_readable import agent_help
53
+ import logging
54
+
55
+ print(agent_help(logging.Logger))
56
+ ```
57
+
58
+ `agent_help()` works on plain Python objects with no setup required. It returns a curated public API list from runtime introspection, so agents see what exists in the installed version instead of guessing from stale training data.
59
+
60
+ Library authors can optionally add usage rules next to the code:
61
+
62
+ ```python
63
+ class Sensor:
64
+ """Reads a value from a hardware sensor."""
65
+
66
+ def read(self) -> float:
67
+ """Read the current sensor value."""
68
+
69
+ @classmethod
70
+ def __agent_notes__(cls) -> str:
71
+ return """
72
+ ## Do
73
+
74
+ - Call `calibrate()` once during setup, before `read()`.
75
+
76
+ ## Do not
77
+
78
+ - Do not call `read()` before calibration on first use.
79
+ """
80
+ ```
81
+
82
+ ## Docs
83
+
84
+ - [Getting started](docs/getting-started.md): installation, quickstart, CLI usage, and other language implementations.
85
+ - [Why it matters](docs/why.md): the API hallucination problem, token efficiency, and how this compares to other agent-doc patterns.
86
+ - [Examples](docs/examples.md): wrapping existing classes, inherited notes, duck typing, plain classes, modules, functions, and methods.
87
+ - [Authoring guide](docs/authoring.md): `__agent_help__`, `__agent_notes__`, class docstring hints, freshness guidance, and API reference.
88
+ - [FAQ](docs/faq.md): common questions about agent skills, docstrings, `AGENTS.md`, third-party libraries, and constrained decoding.
89
+ - [Benchmark](docs/benchmark.md): methodology and harness measuring whether `agent_help()` context reduces hallucinated-API failures.
90
+
91
+ ## Other Languages
92
+
93
+ - TypeScript: [`agent-readable-ts`](https://github.com/zydo/agent-readable-ts)
94
+
95
+ ## License
96
+
97
+ [MIT](LICENSE)
@@ -0,0 +1,70 @@
1
+ # agent-readable
2
+
3
+ [![CI](https://github.com/zydo/agent-readable/actions/workflows/ci.yml/badge.svg)](https://github.com/zydo/agent-readable/actions/workflows/ci.yml) [![PyPI](https://img.shields.io/pypi/v/agent-readable.svg)](https://pypi.org/project/agent-readable/)
4
+
5
+ Stop coding agents from hallucinating Python APIs. `agent_help(target)` reads the live public API surface from any Python class, module, function, or method and renders compact, agent-oriented Markdown with current signatures and docstring summaries. When a library opts in, it can also return class-level usage rules such as lifecycle order, preconditions, and anti-patterns.
6
+
7
+ To let your coding agent automatically call `agent_help()` before using an unfamiliar API, install the companion skill:
8
+
9
+ ```bash
10
+ npx skills add zydo/skills --skill agent-readable
11
+ ```
12
+
13
+ <!-- markdownlint-disable MD033 -->
14
+ <p align="center">
15
+ <strong><code>logging.Logger</code> compared with <code>agent_help()</code> and <code>help()</code></strong><br>
16
+ <img src="https://raw.githubusercontent.com/zydo/agent-readable/main/docs/agent_help_vs_help.gif" alt="agent_help vs help">
17
+ </p>
18
+ <!-- markdownlint-enable MD033 -->
19
+
20
+ ## Quickstart
21
+
22
+ See [Getting started](docs/getting-started.md) for installation, one-off CLI usage with `uvx`, `uv tool run`, or `pipx`, and full CLI examples.
23
+
24
+ ```python
25
+ from agent_readable import agent_help
26
+ import logging
27
+
28
+ print(agent_help(logging.Logger))
29
+ ```
30
+
31
+ `agent_help()` works on plain Python objects with no setup required. It returns a curated public API list from runtime introspection, so agents see what exists in the installed version instead of guessing from stale training data.
32
+
33
+ Library authors can optionally add usage rules next to the code:
34
+
35
+ ```python
36
+ class Sensor:
37
+ """Reads a value from a hardware sensor."""
38
+
39
+ def read(self) -> float:
40
+ """Read the current sensor value."""
41
+
42
+ @classmethod
43
+ def __agent_notes__(cls) -> str:
44
+ return """
45
+ ## Do
46
+
47
+ - Call `calibrate()` once during setup, before `read()`.
48
+
49
+ ## Do not
50
+
51
+ - Do not call `read()` before calibration on first use.
52
+ """
53
+ ```
54
+
55
+ ## Docs
56
+
57
+ - [Getting started](docs/getting-started.md): installation, quickstart, CLI usage, and other language implementations.
58
+ - [Why it matters](docs/why.md): the API hallucination problem, token efficiency, and how this compares to other agent-doc patterns.
59
+ - [Examples](docs/examples.md): wrapping existing classes, inherited notes, duck typing, plain classes, modules, functions, and methods.
60
+ - [Authoring guide](docs/authoring.md): `__agent_help__`, `__agent_notes__`, class docstring hints, freshness guidance, and API reference.
61
+ - [FAQ](docs/faq.md): common questions about agent skills, docstrings, `AGENTS.md`, third-party libraries, and constrained decoding.
62
+ - [Benchmark](docs/benchmark.md): methodology and harness measuring whether `agent_help()` context reduces hallucinated-API failures.
63
+
64
+ ## Other Languages
65
+
66
+ - TypeScript: [`agent-readable-ts`](https://github.com/zydo/agent-readable-ts)
67
+
68
+ ## License
69
+
70
+ [MIT](LICENSE)
@@ -0,0 +1,79 @@
1
+ # Authoring guide
2
+
3
+ ## Keeping agent docs up to date
4
+
5
+ Agent docs can go stale when classes change: new methods, changed behavior, or removed APIs. Install the companion skill to teach your agent to run `agent_help()` before modifying a class, prefer docstrings over `__agent_notes__()` for API summaries, and verify that the output stays accurate after changes.
6
+
7
+ ```bash
8
+ npx skills add zydo/skills --skill agent-readable
9
+ ```
10
+
11
+ ## The `__agent_help__` protocol
12
+
13
+ `__agent_help__()` is a dunder protocol for tool-specific documentation, similar in spirit to:
14
+
15
+ - `__doc__`: read by Python `help()`, Sphinx, IDEs, and REPLs.
16
+ - `__rich_repr__`: read by Rich when it renders an object.
17
+ - `__html__`: read by Django, Jinja2, and MarkupSafe when rendering HTML.
18
+
19
+ Unlike `__str__` or `__fspath__`, these dunders do not change Python runtime behavior. They are metadata slots a specific tool reads when it wants a representation. `__agent_help__` follows the same pattern for agent documentation.
20
+
21
+ Classes that define a `@classmethod` named `__agent_help__` returning a `str` are considered agent-readable. Modules can define a top-level `__agent_help__` attribute, either callable or string. Call the top-level `agent_help(obj)` function to get the docs, just like `str()` calls `__str__()`.
22
+
23
+ The `AgentReadable` `typing.Protocol` and `AgentReadableMixin` are provided for convenience and type-checking, but neither is required.
24
+
25
+ ## `__agent_help__` vs `__agent_notes__`
26
+
27
+ The two dunders intentionally encode different composition rules:
28
+
29
+ | Aspect | `__agent_help__()` | `__agent_notes__()` |
30
+ | --------------- | ------------------------------------------ | ----------------------------------------------------------------------------- |
31
+ | Semantics | Replaces the auto-generated base document | Additive: appended after the auto-doc or custom `__agent_help__()` output |
32
+ | Composition | Single class wins, closest in MRO | Accumulated across the MRO; leaf class wins on conflict |
33
+ | When to use | Custom control over the base document | Extra do/don't rules on top of any base |
34
+ | Skipped when | Always called if defined | Never skipped when defined; a notes method that raises is skipped |
35
+ | Mixin required? | No | No |
36
+
37
+ When a class defines both `__agent_help__()` and `__agent_notes__()`, the custom help replaces the auto-generated base document and the notes are appended after it — the same additive behavior as the auto-doc path, so no combination of the two dunders silently drops notes. If you need full verbatim control of the entire output, do not define `__agent_notes__()` anywhere in the class hierarchy.
38
+
39
+ ## Class docstring hints
40
+
41
+ For classes that inherit `AgentReadableMixin`, add a short hint in the class docstring:
42
+
43
+ ```python
44
+ class ResourcePool(AgentReadableMixin):
45
+ """
46
+ Rotates interchangeable resources such as API keys.
47
+
48
+ Agent usage:
49
+ Run ``agent_help(ResourcePool)`` before using this class in generated code.
50
+ """
51
+ ```
52
+
53
+ This way, even agents that only see the source or call `help()` are reminded to check `agent_help()`.
54
+
55
+ ## API reference
56
+
57
+ ### `agent_help(target)`
58
+
59
+ Render agent-oriented Markdown for a class, module, function, or method.
60
+
61
+ For classes, the output includes purpose, constructor, public API (methods, properties, cached properties, constants, and enum members), default usage rules, and accumulated `__agent_notes__()` sections when present. For modules, it includes purpose, public functions (including C builtins), public classes, and constants. For functions and methods, it includes signature, docstring, and default usage rules.
62
+
63
+ For classes and instances, if `__agent_help__()` is defined through the mixin or duck typing, it is called and its return value is used as the base document. Duck-typed implementations replace the auto-generated base; `__agent_notes__()` sections from the MRO are still appended after it (the mixin default already embeds notes, so its output is used as-is). If `__agent_help__()` raises, `agent_help()` falls back to the auto-generated path, which does include notes. A `__agent_notes__()` that raises is skipped rather than fatal, so one broken notes method cannot take down help for a class and its subclasses.
64
+
65
+ For modules, if the module defines a `__agent_help__` attribute, either callable or string, it is used. Otherwise, auto-generated docs are produced from the module docstring and its public functions and classes. When the module defines `__all__`, that list is the authoritative public API, so re-exported symbols are included. Otherwise public members are discovered by introspection, skipping private names and anything defined outside the module.
66
+
67
+ For functions and methods, if the routine defines a `__agent_help__` attribute, either callable or string, it is used. Otherwise, auto-generated docs render the signature, full docstring, and usage rules. A bound method's signature drops `self`, and a classmethod's signature drops `cls`. If a callable `__agent_help__` raises, `agent_help()` falls back to the auto-generated path.
68
+
69
+ ### `AgentReadable`
70
+
71
+ A runtime-checkable `typing.Protocol` that requires `__agent_help__() -> str`.
72
+
73
+ ### `AgentReadableMixin`
74
+
75
+ A mixin class for classes that provides a default `__agent_help__()` implementation using introspection. The mixin is convenience only: defining `__agent_notes__()` directly on any class also works, and notes are collected automatically regardless of inheritance from the mixin.
76
+
77
+ The mixin does not apply to modules; modules are supported directly by `agent_help()`.
78
+
79
+ If a class inherits from `AgentReadableMixin`, coding agents should call `agent_help(TheClass)` before generating code that uses it.
@@ -0,0 +1,79 @@
1
+ # Benchmark: does `agent_help()` context reduce API hallucination?
2
+
3
+ **Status: not yet run.** This page documents the methodology and the harness
4
+ (`scripts/benchmark.py`). No numbers are published until a real run has been
5
+ executed and recorded here — claims in the README about hallucination and
6
+ token efficiency remain unevidenced until then.
7
+
8
+ ## The question
9
+
10
+ When a coding agent writes Python against an API it has not seen (a new
11
+ library, a recent release, a project-private module), does injecting
12
+ `agent_help(target)` output into the prompt reduce failures compared with the
13
+ same prompt without it?
14
+
15
+ The failure modes scored are the two documented in [Why it matters](why.md):
16
+
17
+ - **What exists** — hallucinated attributes and methods, measured as
18
+ `AttributeError` at runtime.
19
+ - **How to use it** — wrong signatures or call shapes, measured as `TypeError`
20
+ at runtime.
21
+
22
+ ## Method
23
+
24
+ For each task (one target library plus a natural-language coding prompt) and
25
+ each condition, the harness asks the model for a single runnable script and
26
+ executes it against the installed library:
27
+
28
+ | Condition | Prompt |
29
+ | --- | --- |
30
+ | `baseline` | The task prompt only. |
31
+ | `agent_help` | The task prompt plus `agent_help(target)` output labeled as reference documentation for the installed version. |
32
+
33
+ Every generated script runs in a fresh subprocess against the same
34
+ environment. Outcomes:
35
+
36
+ | Outcome | Meaning |
37
+ | --- | --- |
38
+ | `ok` | Script exits 0. |
39
+ | `attributeerror` | Runtime `AttributeError` — most directly indicates a hallucinated API. |
40
+ | `typeerror` | Runtime `TypeError` — wrong signature or call shape. |
41
+ | `nameerror`, `importerror` | Reference or import failures. |
42
+ | `other_error`, `timeout`, `no_code`, `api_error` | Everything else; reported but not counted as API hallucination. |
43
+
44
+ Headline metric: the `ok` rate per condition, plus the split between
45
+ `attributeerror` and `typeerror`. Results are recorded below with the date,
46
+ model, sample count, and library versions.
47
+
48
+ ## Running it
49
+
50
+ The harness calls the Anthropic API (needs `ANTHROPIC_API_KEY` or an
51
+ `ant auth login` profile) and requires the target libraries to be installed.
52
+ It is deliberately not part of CI.
53
+
54
+ ```bash
55
+ uv run --with anthropic --with icalendar --with feedparser \
56
+ python scripts/benchmark.py --samples 5 --out .localonly/benchmark-results.json
57
+ ```
58
+
59
+ Custom task sets are JSON files of `{module, target, task}` entries passed via
60
+ `--tasks`. Choose targets the model plausibly has weak training coverage of:
61
+ small libraries, recent releases, or project-private code — for well-known
62
+ stable APIs the model may succeed in both conditions and the benchmark
63
+ measures nothing.
64
+
65
+ ## Threats to validity
66
+
67
+ - **Target familiarity.** A model that already knows a library reduces the
68
+ gap; a model that misremembers a *changed* API may fail both conditions.
69
+ Unfamiliar targets are the population of interest.
70
+ - **Task wording.** The injected documentation is the only intended
71
+ difference; prompts are otherwise identical.
72
+ - **Execution-only scoring.** A script can exit 0 and still be semantically
73
+ wrong; `ok` is an upper bound on correctness, not a proof of it.
74
+ - **Sample size.** Per-task differences at small sample counts are noise;
75
+ read the aggregate.
76
+
77
+ ## Results
78
+
79
+ _None yet._