librarian-a2a 0.10.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- librarian_a2a-0.10.0/LICENSE +202 -0
- librarian_a2a-0.10.0/PKG-INFO +418 -0
- librarian_a2a-0.10.0/README.md +377 -0
- librarian_a2a-0.10.0/pyproject.toml +128 -0
- librarian_a2a-0.10.0/pyproject.toml.orig +101 -0
- librarian_a2a-0.10.0/src/librarian/__init__.py +3 -0
- librarian_a2a-0.10.0/src/librarian/approvals.py +551 -0
- librarian_a2a-0.10.0/src/librarian/audit.py +64 -0
- librarian_a2a-0.10.0/src/librarian/calls.py +138 -0
- librarian_a2a-0.10.0/src/librarian/cli.py +499 -0
- librarian_a2a-0.10.0/src/librarian/config.py +364 -0
- librarian_a2a-0.10.0/src/librarian/credentials.py +479 -0
- librarian_a2a-0.10.0/src/librarian/docs.py +573 -0
- librarian_a2a-0.10.0/src/librarian/drift.py +200 -0
- librarian_a2a-0.10.0/src/librarian/executor.py +553 -0
- librarian_a2a-0.10.0/src/librarian/finddocs.py +376 -0
- librarian_a2a-0.10.0/src/librarian/github_app.py +287 -0
- librarian_a2a-0.10.0/src/librarian/mcp_server.py +201 -0
- librarian_a2a-0.10.0/src/librarian/net.py +77 -0
- librarian_a2a-0.10.0/src/librarian/notifiers.py +444 -0
- librarian_a2a-0.10.0/src/librarian/observations.py +301 -0
- librarian_a2a-0.10.0/src/librarian/openapi.py +395 -0
- librarian_a2a-0.10.0/src/librarian/opsearch.py +141 -0
- librarian_a2a-0.10.0/src/librarian/planner.py +287 -0
- librarian_a2a-0.10.0/src/librarian/policy.py +181 -0
- librarian_a2a-0.10.0/src/librarian/principals.py +143 -0
- librarian_a2a-0.10.0/src/librarian/publish.py +227 -0
- librarian_a2a-0.10.0/src/librarian/py.typed +0 -0
- librarian_a2a-0.10.0/src/librarian/recipes.py +159 -0
- librarian_a2a-0.10.0/src/librarian/registry/__init__.py +35 -0
- librarian_a2a-0.10.0/src/librarian/registry/models.py +399 -0
- librarian_a2a-0.10.0/src/librarian/registry/store.py +264 -0
- librarian_a2a-0.10.0/src/librarian/secrets.py +363 -0
- librarian_a2a-0.10.0/src/librarian/server.py +689 -0
- librarian_a2a-0.10.0/src/librarian/server_state.py +4 -0
- librarian_a2a-0.10.0/src/librarian/service.py +1207 -0
- librarian_a2a-0.10.0/src/librarian/shaping.py +76 -0
- librarian_a2a-0.10.0/src/librarian/state.py +217 -0
- librarian_a2a-0.10.0/src/librarian/telemetry.py +88 -0
- librarian_a2a-0.10.0/src/librarian/web.py +321 -0
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright [yyyy] [name of copyright owner]
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
|
@@ -0,0 +1,418 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: librarian-a2a
|
|
3
|
+
Version: 0.10.0
|
|
4
|
+
Summary: An A2A agent that learns APIs on demand and gives other agents safe, audited access: reads within policy, writes with human approval.
|
|
5
|
+
Keywords: a2a,agents,api,openapi,llm,pydantic-ai
|
|
6
|
+
Author: kultar
|
|
7
|
+
Author-email: kultar <kultar@kultar.net>
|
|
8
|
+
License-Expression: Apache-2.0
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Classifier: Development Status :: 3 - Alpha
|
|
11
|
+
Classifier: Intended Audience :: Developers
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
14
|
+
Classifier: Typing :: Typed
|
|
15
|
+
Requires-Dist: a2a-sdk[http-server,sqlite]>=1.2.2
|
|
16
|
+
Requires-Dist: httpx>=0.28.1
|
|
17
|
+
Requires-Dist: mcp>=2.3
|
|
18
|
+
Requires-Dist: opentelemetry-api>=1.33
|
|
19
|
+
Requires-Dist: pydantic>=2.14.0
|
|
20
|
+
Requires-Dist: pydantic-ai-slim[anthropic,openai]>=2.55.0
|
|
21
|
+
Requires-Dist: pydantic-settings>=2.15.0
|
|
22
|
+
Requires-Dist: pyjwt[crypto]>=2.9
|
|
23
|
+
Requires-Dist: python-multipart>=0.0.32
|
|
24
|
+
Requires-Dist: pyyaml>=6.0.3
|
|
25
|
+
Requires-Dist: typer>=0.27.3
|
|
26
|
+
Requires-Dist: uvicorn>=0.54.0
|
|
27
|
+
Requires-Dist: boto3>=1.35 ; extra == 'aws'
|
|
28
|
+
Requires-Dist: opentelemetry-exporter-otlp-proto-http>=1.33 ; extra == 'otel'
|
|
29
|
+
Requires-Dist: opentelemetry-instrumentation-asgi>=0.54b0 ; extra == 'otel'
|
|
30
|
+
Requires-Dist: opentelemetry-sdk>=1.33 ; extra == 'otel'
|
|
31
|
+
Requires-Dist: a2a-sdk[postgresql]>=1.2.2 ; extra == 'postgres'
|
|
32
|
+
Requires-Python: >=3.12
|
|
33
|
+
Project-URL: Homepage, https://github.com/kultar/librarian
|
|
34
|
+
Project-URL: Repository, https://github.com/kultar/librarian
|
|
35
|
+
Project-URL: Issues, https://github.com/kultar/librarian/issues
|
|
36
|
+
Project-URL: Security, https://github.com/kultar/librarian/security/policy
|
|
37
|
+
Provides-Extra: aws
|
|
38
|
+
Provides-Extra: otel
|
|
39
|
+
Provides-Extra: postgres
|
|
40
|
+
Description-Content-Type: text/markdown
|
|
41
|
+
|
|
42
|
+
# librarian
|
|
43
|
+
|
|
44
|
+
**An A2A agent that learns APIs on demand and gives other agents safe, audited access to them: reads freely within policy, writes only with human approval.**
|
|
45
|
+
|
|
46
|
+
Agents shouldn't each carry their own Jira token, GitHub token and API client. With librarian,
|
|
47
|
+
agents ask one specialist, over [A2A](https://a2a-protocol.org), for the data they need. The
|
|
48
|
+
librarian learns the API the first time it is needed (from its OpenAPI spec, or from its
|
|
49
|
+
documentation when there is no spec), asks a human for a
|
|
50
|
+
credential (once, out of band), and from then on serves every agent, without ever handing the
|
|
51
|
+
credential out.
|
|
52
|
+
|
|
53
|
+
```
|
|
54
|
+
your agents ──A2A──► librarian ──(policy engine)──► Jira, GitHub, internal APIs, ...
|
|
55
|
+
│
|
|
56
|
+
├── registry (git): what each API is and which operations are reads
|
|
57
|
+
├── secret store: credentials (agents only ever see references)
|
|
58
|
+
└── planner (Pydantic AI, optional) ──► any model / LiteLLM
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
> Status: **alpha (v0.10)**. Expect breaking changes.
|
|
62
|
+
|
|
63
|
+
## Why
|
|
64
|
+
|
|
65
|
+
- **One choke point.** Every call from every agent goes through one deterministic policy engine and
|
|
66
|
+
lands in one audit log.
|
|
67
|
+
- **Credentials never travel through agents.** Secrets enter via a one-time secure page, straight
|
|
68
|
+
into the secret store. Agents, messages, logs and the LLM only see references like `jira/readonly`.
|
|
69
|
+
- **Learn once, reuse everywhere.** The first agent that needs an API triggers onboarding; every
|
|
70
|
+
agent after that benefits.
|
|
71
|
+
- **The LLM proposes, code decides.** The optional planner can suggest any call, but only the rule
|
|
72
|
+
engine (no LLM) can approve it. No prompt can talk its way into a write.
|
|
73
|
+
|
|
74
|
+
## Get started
|
|
75
|
+
|
|
76
|
+
| I want to... | Go to |
|
|
77
|
+
|---|---|
|
|
78
|
+
| Try it on my laptop (10 minutes, Docker) | [docs/getting-started.md](docs/getting-started.md) |
|
|
79
|
+
| Run it for my team (Kubernetes, step by step) | [docs/deploy.md](docs/deploy.md) |
|
|
80
|
+
| Connect agents (LiteLLM, MCP clients, A2A) | [docs/integrations.md](docs/integrations.md) |
|
|
81
|
+
| Fix something | [docs/troubleshooting.md](docs/troubleshooting.md) |
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
pip install librarian-a2a # the CLI and import are `librarian`
|
|
85
|
+
docker pull ghcr.io/kultar/librarian:0.10.0
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
### What happens on a request
|
|
89
|
+
|
|
90
|
+
1. An agent asks for data from an API librarian doesn't know yet.
|
|
91
|
+
2. librarian learns it: from its OpenAPI spec, else its documentation (found by itself if
|
|
92
|
+
needed), else the model's knowledge. The API is usable for reads at once, and a pull request
|
|
93
|
+
is opened on your registry repo for a person to review.
|
|
94
|
+
3. If the API needs a key, the agent's task waits and approvers get a Slack card. Someone follows
|
|
95
|
+
the link, signs in and adds the key (it goes straight to your secret store), and the task
|
|
96
|
+
carries on by itself.
|
|
97
|
+
4. Writes wait for a person's approval every time, unless a narrow, expiring grant covers them.
|
|
98
|
+
|
|
99
|
+
Nothing is set up per API in advance: the registry fills itself as agents need things.
|
|
100
|
+
|
|
101
|
+
## Skills (A2A)
|
|
102
|
+
|
|
103
|
+
| Skill | What it does | LLM? |
|
|
104
|
+
|---|---|---|
|
|
105
|
+
| `execute` | Run one read operation: `{"skill":"execute","api":"jira","operation":"getIssue","args":{"issueIdOrKey":"ABC-1"}}`. Optional `select` (dotted paths, e.g. `["fields.summary"]`) and `max_pages` | No |
|
|
106
|
+
| `capabilities` | List APIs and read operations: `{"skill":"capabilities","query":"issues"}` | No |
|
|
107
|
+
| `onboard` | Learn an API: `{"skill":"onboard","api":"jira","spec_url":"https://..."}`, `"base_url"` to auto-discover, or `"docs_url"` + `"base_url"` when there is no spec | No |
|
|
108
|
+
| `write` | Change data, with human approval or a grant: `{"skill":"write","api":"jira","operation":"addComment","args":{...},"body":{...},"reason":"why"}` | No |
|
|
109
|
+
| `fetch` | Plain language: *"open bugs assigned to the payments team"* (only when a model is configured) | Yes |
|
|
110
|
+
|
|
111
|
+
The same skills are available as **MCP tools** at `/mcp` (Streamable HTTP, same caller keys):
|
|
112
|
+
`capabilities`, `execute`, `fetch`, `write`, `write_status`, `onboard`. MCP calls never wait on a
|
|
113
|
+
human: writes return `pending_approval` and librarian finishes them once approved. librarian
|
|
114
|
+
works behind the **LiteLLM** gateway as both an MCP server and an A2A agent
|
|
115
|
+
([docs/integrations.md](docs/integrations.md#litellm-agent-gateway)).
|
|
116
|
+
|
|
117
|
+
Results are A2A artifacts with the data plus **provenance**: which API and operation, the URL
|
|
118
|
+
(secrets redacted), when, for which calling agent and principal, and which credential reference.
|
|
119
|
+
|
|
120
|
+
When a credential is missing, the task enters A2A's `TASK_STATE_AUTH_REQUIRED`, approvers are
|
|
121
|
+
notified (Slack by default), and the task **resumes on its own** once someone adds the key. The
|
|
122
|
+
caller polls or subscribes to the task as usual.
|
|
123
|
+
|
|
124
|
+
## Configuration
|
|
125
|
+
|
|
126
|
+
Settings come from a YAML file (`--config`) and/or `LIBRARIAN_*` environment variables (env wins;
|
|
127
|
+
nested keys use `__`, e.g. `LIBRARIAN_NOTIFIER__SLACK__BOT_TOKEN`). See
|
|
128
|
+
[`examples/librarian.yaml`](examples/librarian.yaml) for every option.
|
|
129
|
+
|
|
130
|
+
**Models / LiteLLM.** `fetch` needs a model. Use any Pydantic AI model string
|
|
131
|
+
(`model: anthropic:claude-sonnet-5-5`, with the provider's usual API-key env var), or point at any
|
|
132
|
+
OpenAI-compatible gateway such as LiteLLM:
|
|
133
|
+
|
|
134
|
+
```yaml
|
|
135
|
+
model: claude-sonnet-5-5 # the model name your gateway serves
|
|
136
|
+
model_base_url: http://litellm:4000/v1
|
|
137
|
+
model_api_key: sk-litellm-... # or LIBRARIAN_MODEL_API_KEY
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
**Secrets.** Choose a backend with `secrets_backend`:
|
|
141
|
+
|
|
142
|
+
- `file` (default): one `0600` file per secret, good locally or on a mounted volume.
|
|
143
|
+
- `aws`: AWS Secrets Manager (`uv sync --extra aws`). `jira/readonly` maps to the secret
|
|
144
|
+
`<prefix>jira/readonly`; append `#key` to read one key of a JSON secret
|
|
145
|
+
(`service/jira#api_token`). Values are cached briefly (`aws_secrets.cache_seconds`).
|
|
146
|
+
|
|
147
|
+
Environment variables are always checked last (`jira/readonly` → `LIBRARIAN_SECRET_JIRA_READONLY`).
|
|
148
|
+
Internal secrets (caller keys, the link-signing key) live under the reserved `librarian/` prefix.
|
|
149
|
+
|
|
150
|
+
**State.** Credential requests, approvals, A2A tasks and APIs learned on demand are stored in
|
|
151
|
+
SQLite by default (`.librarian/librarian.db`), so they survive restarts. For several replicas use
|
|
152
|
+
Postgres (`database_url: postgresql+asyncpg://...`, the `postgres` extra; in the image) and
|
|
153
|
+
`secrets_backend: aws`: then every replica shares everything, an API learned through one is usable
|
|
154
|
+
through the others at once, and pods need no volume. Set `replicas` (the Helm chart does) and
|
|
155
|
+
librarian refuses to start with per-replica state. Rate limits are per replica.
|
|
156
|
+
|
|
157
|
+
**Caller authentication.** List agents and the secret holding each one's key:
|
|
158
|
+
|
|
159
|
+
```yaml
|
|
160
|
+
callers:
|
|
161
|
+
- agent: release-notes-agent
|
|
162
|
+
key_ref: librarian/callers/release-notes-agent # internal secrets live under librarian/
|
|
163
|
+
apis: ["jira", "github-*"] # glob patterns; default ["*"]
|
|
164
|
+
can_onboard: false # may this agent teach new APIs?
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
An agent only sees and uses the APIs its patterns allow, in `capabilities`, `execute` and `fetch`.
|
|
168
|
+
|
|
169
|
+
With no callers configured the server is open: fine on a laptop, never in a shared deployment.
|
|
170
|
+
|
|
171
|
+
## Observability
|
|
172
|
+
|
|
173
|
+
Set `otel_enabled: true` (install the `otel` extra; the Docker image has it) and the standard
|
|
174
|
+
`OTEL_EXPORTER_OTLP_ENDPOINT`. You get:
|
|
175
|
+
|
|
176
|
+
- **Traces**: `librarian.skill` and `librarian.call` spans (plus the A2A SDK's and Pydantic AI's
|
|
177
|
+
model-call spans). A calling agent's `traceparent` is honoured, so its trace continues through
|
|
178
|
+
the librarian. Trace headers are **never** forwarded to third-party APIs.
|
|
179
|
+
- **Metrics**: `librarian.api.calls` (by API, operation, caller, status class),
|
|
180
|
+
`librarian.api.call.duration`, `librarian.policy.denials`, `librarian.credential.requests`.
|
|
181
|
+
|
|
182
|
+
The JSONL audit log is separate and always on.
|
|
183
|
+
|
|
184
|
+
## Big APIs
|
|
185
|
+
|
|
186
|
+
- **Operation search.** Requests are matched to operations with BM25 ranking over operation ids
|
|
187
|
+
(split into words: `getIssueWatchers` -> issue, watch), tags, paths and summaries. On Jira's
|
|
188
|
+
620-operation API, "who is watching an issue" finds `getIssueWatchers` first.
|
|
189
|
+
- **Glossary.** Registry entries can declare `discovery` reads that teach instance-specific
|
|
190
|
+
names, e.g. Jira's `/field` maps `customfield_10020` to "Sprint". They run after a credential
|
|
191
|
+
is approved (or `librarian discover <api>`), and the planner sees the relevant entries.
|
|
192
|
+
- **Recipes.** A successful plain-language request is saved as a template: "Is PAY-1 open?"
|
|
193
|
+
becomes `Is {p0} open`, and "Is PAY-7 open?" later replays the same calls **without the LLM**
|
|
194
|
+
(still policy-checked, with the caller's permissions). Plans whose later calls use values from
|
|
195
|
+
earlier results are never saved. `librarian recipes list|delete`; `recipes_enabled: false`
|
|
196
|
+
turns it off.
|
|
197
|
+
|
|
198
|
+
## APIs without a spec, and APIs that change
|
|
199
|
+
|
|
200
|
+
**Learning without a spec** (needs `model`). librarian learns from, in order of preference:
|
|
201
|
+
|
|
202
|
+
1. **An OpenAPI spec**: given, found at a well-known path, or linked from the docs.
|
|
203
|
+
2. **Documentation pages**: the model reads the page and related pages on the same site and
|
|
204
|
+
lists the operations, each with a verbatim quote. librarian checks every quote against the
|
|
205
|
+
fetched page and drops operations it can't find there. With only a `base_url`, librarian
|
|
206
|
+
**finds the docs itself**: the usual places first (`/llms.txt`, `/docs`, `docs.<domain>`,
|
|
207
|
+
links on the homepage, one click further), then a **web search** if configured.
|
|
208
|
+
3. **The model's own knowledge** (nothing found): reads only, for well-known APIs.
|
|
209
|
+
|
|
210
|
+
Search results are untrusted (anyone can publish "Acme API docs"), so sources are ranked:
|
|
211
|
+
the API's own domain first, then well-known docs hosts (GitHub, ReadMe, GitBook, ...);
|
|
212
|
+
other sites are skipped unless `learning.search.allow_third_party: true`. The review PR says
|
|
213
|
+
how the docs were found and how far they are trusted.
|
|
214
|
+
|
|
215
|
+
```yaml
|
|
216
|
+
learning:
|
|
217
|
+
search:
|
|
218
|
+
provider: model # off | model | searxng | brave
|
|
219
|
+
# url: https://searx.internal (searxng)
|
|
220
|
+
# api_key: via LIBRARIAN_LEARNING__SEARCH__API_KEY (brave)
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
`provider: model` uses the model's own web search. Through a gateway such as LiteLLM that
|
|
224
|
+
needs the Anthropic API format (`model_api: anthropic`); with the OpenAI formats the search is
|
|
225
|
+
refused or silently dropped by the gateway, and librarian reports that instead of guessing.
|
|
226
|
+
|
|
227
|
+
Documentation is untrusted, so the model never decides anything that matters: it has no tools;
|
|
228
|
+
the API host is always the `base_url` you gave (never one from the docs); read vs write is decided
|
|
229
|
+
by HTTP method; the auth *type* may come from the docs but where the key is stored never does.
|
|
230
|
+
Operations from docs or model knowledge are **unconfirmed** until a real call succeeds (callers
|
|
231
|
+
see a warning). One that keeps answering 404 is **suspended** and its owners are told.
|
|
232
|
+
|
|
233
|
+
```bash
|
|
234
|
+
uv run librarian learn dogs --docs-url https://dog.ceo/dog-api/documentation/ \
|
|
235
|
+
--base-url https://dog.ceo/api --name "Dog CEO"
|
|
236
|
+
```
|
|
237
|
+
|
|
238
|
+
**Noticing changes.** Every `drift.recheck_hours` (default 24), one replica re-reads each entry's
|
|
239
|
+
spec or docs. A new model read only happens when the docs' text actually changed.
|
|
240
|
+
|
|
241
|
+
| Change | Provisional entry | Verified entry |
|
|
242
|
+
|---|---|---|
|
|
243
|
+
| New operations, optional parameters | updated, re-proposed | proposed for review |
|
|
244
|
+
| Removed operation, new required parameter, moved path | updated, re-proposed | callers warned; writes **suspended**; proposed |
|
|
245
|
+
| HTTP method changed (a read may now change data) | updated, re-proposed | **suspended**; proposed |
|
|
246
|
+
|
|
247
|
+
Verified entries are never changed automatically. The proposal PR keeps reviewed decisions
|
|
248
|
+
(writes enabled, reviewed reads, pagination); merging it lifts the suspensions it addresses.
|
|
249
|
+
|
|
250
|
+
Calls also watch for change: `Deprecation`/`Sunset` headers, repeated 404/405/410 responses, and
|
|
251
|
+
responses that lose most of the fields seen before are reported to the API's owners (once), and
|
|
252
|
+
callers see a warning. `librarian registry recheck [api]` runs a check now;
|
|
253
|
+
`librarian registry clear <api> <operation>` lifts a suspension after checking by hand.
|
|
254
|
+
|
|
255
|
+
## Verified principals
|
|
256
|
+
|
|
257
|
+
By default the person an agent acts for (`principal`) is whatever the agent claims. Turn on
|
|
258
|
+
verification to require proof: the agent sends a token from your identity provider in the
|
|
259
|
+
`X-Principal-Token` header, and librarian checks its signature (issuer's published keys), issuer,
|
|
260
|
+
audience and expiry.
|
|
261
|
+
|
|
262
|
+
```yaml
|
|
263
|
+
principals:
|
|
264
|
+
mode: required # off | optional | required
|
|
265
|
+
issuers:
|
|
266
|
+
- issuer: https://acme.okta.com/oauth2/default
|
|
267
|
+
audience: librarian # tokens must be minted for librarian
|
|
268
|
+
claim: email
|
|
269
|
+
```
|
|
270
|
+
|
|
271
|
+
- `optional`: verified when a token is present; a claimed principal without one is recorded
|
|
272
|
+
but does **not** count as the requester for write approvals.
|
|
273
|
+
- `required`: a claimed principal without a valid token is refused. Requests with no person
|
|
274
|
+
behind them (autonomous agents) still work.
|
|
275
|
+
|
|
276
|
+
Verified identities appear in provenance (`principal_verified: true`) and the audit log. How an
|
|
277
|
+
agent gets a token for librarian on a user's behalf is covered in
|
|
278
|
+
[docs/integrations.md](docs/integrations.md#verified-principals).
|
|
279
|
+
|
|
280
|
+
## Slack notifications
|
|
281
|
+
|
|
282
|
+
1. Create a Slack app from [`deploy/slack-app-manifest.yaml`](deploy/slack-app-manifest.yaml) and
|
|
283
|
+
install it to your workspace.
|
|
284
|
+
2. Set `notifier.type: slack`, the bot token (`LIBRARIAN_NOTIFIER__SLACK__BOT_TOKEN`), a channel,
|
|
285
|
+
and optionally an approver user-group ID (rendered as a real @mention).
|
|
286
|
+
|
|
287
|
+
Approvers get one card per API (deduplicated, with a live "waiting" count) and a **Review request**
|
|
288
|
+
button that opens the secure page: *add a key*, *use an existing secret*, or *deny*. The card
|
|
289
|
+
updates in place and the outcome is posted in the thread. The key itself never goes through Slack.
|
|
290
|
+
|
|
291
|
+
No Slack? `notifier.type: webhook` posts JSON events you can route anywhere.
|
|
292
|
+
|
|
293
|
+
## Writes
|
|
294
|
+
|
|
295
|
+
Reads are the default; writes are opt-in at every level:
|
|
296
|
+
|
|
297
|
+
1. A reviewer marks the operation `writable: true` on a **verified** entry and sets a separate
|
|
298
|
+
write credential (`write_auth`). The approver adding a key can also tick *"allow approved
|
|
299
|
+
writes with this key"*.
|
|
300
|
+
2. The agent is allowed to request writes on that API (`write_apis` in its caller config).
|
|
301
|
+
3. Each write is **approved by a human** for that exact request, or matches a **grant**.
|
|
302
|
+
|
|
303
|
+
The approval card (Slack) shows the operation, arguments and full body. On the approval page the
|
|
304
|
+
approver can **approve once**, **approve and allow similar writes** (a grant for that agent and
|
|
305
|
+
operation, with the argument values they choose kept fixed, an expiry and an optional use cap), or
|
|
306
|
+
**deny**. DELETE operations can only ever be approved once. The task waits in `AUTH_REQUIRED` and
|
|
307
|
+
completes on its own after the decision.
|
|
308
|
+
|
|
309
|
+
Who may approve (`writes.approvers`): the **requester** (the person the agent acts for, or the
|
|
310
|
+
agent's `owner`), the **approver group**, or either (default). Approvers sign in through your
|
|
311
|
+
SSO proxy (`approvers.mode: trusted_header`); agents can never approve.
|
|
312
|
+
|
|
313
|
+
**Autonomous agents** (no person behind them) get an `owner`, who stands in as requester. For
|
|
314
|
+
unattended work, configure bounded grants:
|
|
315
|
+
|
|
316
|
+
```yaml
|
|
317
|
+
callers:
|
|
318
|
+
- agent: triage-bot
|
|
319
|
+
key_ref: librarian/callers/triage-bot
|
|
320
|
+
owner: ops@example.com
|
|
321
|
+
write_apis: ["jira"]
|
|
322
|
+
writes:
|
|
323
|
+
approvers: requester_or_group
|
|
324
|
+
grants:
|
|
325
|
+
- agent: triage-bot
|
|
326
|
+
api: jira
|
|
327
|
+
operation: addLabel
|
|
328
|
+
pinned_args: {} # e.g. {issueIdOrKey: PAY-1}; {} allows any issue
|
|
329
|
+
max_uses_per_day: 50
|
|
330
|
+
notify_owner: true # FYI to the channel on each use
|
|
331
|
+
```
|
|
332
|
+
|
|
333
|
+
An agent with no owner and no person can only be approved by the approver group; if no one could
|
|
334
|
+
approve, the write is refused immediately. `librarian grants list|revoke` and `librarian approvals
|
|
335
|
+
list` manage it all; revoking is the kill switch.
|
|
336
|
+
|
|
337
|
+
## The registry
|
|
338
|
+
|
|
339
|
+
One YAML file per API, normally in a git repo you review by pull request. Onboarded entries start
|
|
340
|
+
as **provisional**:
|
|
341
|
+
|
|
342
|
+
- GET/HEAD operations are reads and usable immediately.
|
|
343
|
+
- POST operations that look like searches are marked `unknown` and blocked until a human sets
|
|
344
|
+
`access: read` + `read_basis: reviewed` **and** the entry is `verified`.
|
|
345
|
+
- Everything else is a write: refused unless a reviewer enables it (see [Writes](#writes)).
|
|
346
|
+
|
|
347
|
+
**Review loop.** With `publish.type: github`, every learned API is also opened as a pull request
|
|
348
|
+
against your registry repo, with a review checklist and the operations that need a decision. Run a
|
|
349
|
+
[git-sync](https://github.com/kubernetes/git-sync) sidecar (or any process that updates the
|
|
350
|
+
registry directory) and merged changes such as `status: verified` are picked up automatically:
|
|
351
|
+
the registry hot-reloads when files change.
|
|
352
|
+
|
|
353
|
+
```yaml
|
|
354
|
+
publish:
|
|
355
|
+
type: github
|
|
356
|
+
repo: acme/api-registry
|
|
357
|
+
directory: registry
|
|
358
|
+
github_app: # recommended: owned by the org, short-lived tokens
|
|
359
|
+
app_id: "123456"
|
|
360
|
+
private_key_file: .librarian/github-app.pem
|
|
361
|
+
# or token: via LIBRARIAN_PUBLISH__TOKEN (contents:write + pull_requests:write)
|
|
362
|
+
```
|
|
363
|
+
|
|
364
|
+
`librarian github-app create --repo acme/api-registry` creates that GitHub App in one step:
|
|
365
|
+
one click on GitHub, install it on the repo, and it saves the key and prints the settings.
|
|
366
|
+
The same App lets git-sync read a private registry repo (see the Helm chart README).
|
|
367
|
+
|
|
368
|
+
When learning an API, the librarian also looks for a cheap "who am I" read (`/me`, `/myself`,
|
|
369
|
+
`/user`, ...) and uses it to test every new credential.
|
|
370
|
+
|
|
371
|
+
Validate in CI with `uv run librarian registry validate`. Format reference:
|
|
372
|
+
[docs/registry.md](docs/registry.md). Example: [`examples/registry/jira.yaml`](examples/registry/jira.yaml).
|
|
373
|
+
|
|
374
|
+
## Security
|
|
375
|
+
|
|
376
|
+
Read [docs/threat-model.md](docs/threat-model.md). Highlights: deterministic policy engine;
|
|
377
|
+
host-locked URL building (callers can't choose hosts); same-origin-only redirects; response size
|
|
378
|
+
caps; SSRF protection for onboarding and provisional entries; one-time, expiring approval links;
|
|
379
|
+
secrets never logged or returned. Report vulnerabilities via [SECURITY.md](SECURITY.md).
|
|
380
|
+
|
|
381
|
+
## Development
|
|
382
|
+
|
|
383
|
+
```bash
|
|
384
|
+
uv sync
|
|
385
|
+
uv run ruff check . && uv run ruff format --check .
|
|
386
|
+
uv run pyright # strict mode
|
|
387
|
+
uv run pytest
|
|
388
|
+
```
|
|
389
|
+
|
|
390
|
+
## Roadmap
|
|
391
|
+
|
|
392
|
+
- **v0.2** ✅: durable state (SQLite/Postgres) and stable approval links; per-agent API
|
|
393
|
+
allowlists; 401 vs 403 handling; DNS-rebinding-proof SSRF guard; GitHub review PRs + hot reload;
|
|
394
|
+
AWS Secrets Manager; field selection and pagination styles; OpenTelemetry
|
|
395
|
+
- **v0.3** ✅: writes behind human approval; grants (approve-and-allow-similar, config grants for
|
|
396
|
+
autonomous agents); agent owners; planner write proposals; grants/approvals CLI
|
|
397
|
+
- **v0.4** ✅: Helm chart (oauth2-proxy sign-in, git-sync registry, hardened pod), separate
|
|
398
|
+
pages listener, registry overlay, release workflow (GHCR, PyPI), example client, LiteLLM compose
|
|
399
|
+
- **v0.5** ✅: verified principals (signed tokens from your identity provider; optional and
|
|
400
|
+
required modes); claimed principals no longer count as requesters once verification is on
|
|
401
|
+
- **v0.6** ✅: big APIs: ranked operation search, discovery-built glossary (e.g. Jira custom
|
|
402
|
+
fields), recipes replayed without the LLM
|
|
403
|
+
- **v0.7** ✅: MCP endpoint; LiteLLM agent gateway (MCP + A2A); exactly-once writes across
|
|
404
|
+
replicas; integration suite
|
|
405
|
+
- **v0.8** ✅: APIs without a spec (documentation, model knowledge) with quote-checked
|
|
406
|
+
extraction and confirm-on-use; change detection (spec/docs re-checks, deprecation and
|
|
407
|
+
removal signals, response shape) with suspensions and review proposals
|
|
408
|
+
- **v0.9** ✅: finding documentation from just a base URL (usual places, then web search via
|
|
409
|
+
the model or SearXNG/Brave), trust-ranked sources; Anthropic API format for gateways
|
|
410
|
+
- **v0.10** ✅: GitHub App for the registry repo (review PRs and git-sync), created in one step
|
|
411
|
+
with `librarian github-app create`; several replicas with no volumes (learned APIs in Postgres,
|
|
412
|
+
PodDisruptionBudget, spread across nodes); step-by-step docs
|
|
413
|
+
- **Later**: Vault/GCP/Azure secret stores; per-user OAuth; per-API approver routing in Slack;
|
|
414
|
+
community registry with signed entries
|
|
415
|
+
|
|
416
|
+
## License
|
|
417
|
+
|
|
418
|
+
[Apache-2.0](LICENSE)
|