avowa-runtime 0.11.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- avowa_runtime-0.11.0/LICENSE +202 -0
- avowa_runtime-0.11.0/NOTICE +4 -0
- avowa_runtime-0.11.0/PKG-INFO +358 -0
- avowa_runtime-0.11.0/README.md +337 -0
- avowa_runtime-0.11.0/avowa_runtime/__init__.py +1613 -0
- avowa_runtime-0.11.0/avowa_runtime/adapters.py +181 -0
- avowa_runtime-0.11.0/avowa_runtime/auto.py +327 -0
- avowa_runtime-0.11.0/avowa_runtime/bench.py +119 -0
- avowa_runtime-0.11.0/avowa_runtime/bootstrap/sitecustomize.py +36 -0
- avowa_runtime-0.11.0/avowa_runtime/langchain.py +102 -0
- avowa_runtime-0.11.0/avowa_runtime/run.py +139 -0
- avowa_runtime-0.11.0/avowa_runtime.egg-info/PKG-INFO +358 -0
- avowa_runtime-0.11.0/avowa_runtime.egg-info/SOURCES.txt +16 -0
- avowa_runtime-0.11.0/avowa_runtime.egg-info/dependency_links.txt +1 -0
- avowa_runtime-0.11.0/avowa_runtime.egg-info/entry_points.txt +2 -0
- avowa_runtime-0.11.0/avowa_runtime.egg-info/top_level.txt +1 -0
- avowa_runtime-0.11.0/pyproject.toml +37 -0
- avowa_runtime-0.11.0/setup.cfg +4 -0
|
@@ -0,0 +1,202 @@
|
|
|
1
|
+
|
|
2
|
+
Apache License
|
|
3
|
+
Version 2.0, January 2004
|
|
4
|
+
http://www.apache.org/licenses/
|
|
5
|
+
|
|
6
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
7
|
+
|
|
8
|
+
1. Definitions.
|
|
9
|
+
|
|
10
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
11
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
12
|
+
|
|
13
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
14
|
+
the copyright owner that is granting the License.
|
|
15
|
+
|
|
16
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
17
|
+
other entities that control, are controlled by, or are under common
|
|
18
|
+
control with that entity. For the purposes of this definition,
|
|
19
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
20
|
+
direction or management of such entity, whether by contract or
|
|
21
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
22
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
23
|
+
|
|
24
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
25
|
+
exercising permissions granted by this License.
|
|
26
|
+
|
|
27
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
28
|
+
including but not limited to software source code, documentation
|
|
29
|
+
source, and configuration files.
|
|
30
|
+
|
|
31
|
+
"Object" form shall mean any form resulting from mechanical
|
|
32
|
+
transformation or translation of a Source form, including but
|
|
33
|
+
not limited to compiled object code, generated documentation,
|
|
34
|
+
and conversions to other media types.
|
|
35
|
+
|
|
36
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
37
|
+
Object form, made available under the License, as indicated by a
|
|
38
|
+
copyright notice that is included in or attached to the work
|
|
39
|
+
(an example is provided in the Appendix below).
|
|
40
|
+
|
|
41
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
42
|
+
form, that is based on (or derived from) the Work and for which the
|
|
43
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
44
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
45
|
+
of this License, Derivative Works shall not include works that remain
|
|
46
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
47
|
+
the Work and Derivative Works thereof.
|
|
48
|
+
|
|
49
|
+
"Contribution" shall mean any work of authorship, including
|
|
50
|
+
the original version of the Work and any modifications or additions
|
|
51
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
52
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
53
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
54
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
55
|
+
means any form of electronic, verbal, or written communication sent
|
|
56
|
+
to the Licensor or its representatives, including but not limited to
|
|
57
|
+
communication on electronic mailing lists, source code control systems,
|
|
58
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
59
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
60
|
+
excluding communication that is conspicuously marked or otherwise
|
|
61
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
62
|
+
|
|
63
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
64
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
65
|
+
subsequently incorporated within the Work.
|
|
66
|
+
|
|
67
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
68
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
69
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
70
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
71
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
72
|
+
Work and such Derivative Works in Source or Object form.
|
|
73
|
+
|
|
74
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
75
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
76
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
77
|
+
(except as stated in this section) patent license to make, have made,
|
|
78
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
79
|
+
where such license applies only to those patent claims licensable
|
|
80
|
+
by such Contributor that are necessarily infringed by their
|
|
81
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
82
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
83
|
+
institute patent litigation against any entity (including a
|
|
84
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
85
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
86
|
+
or contributory patent infringement, then any patent licenses
|
|
87
|
+
granted to You under this License for that Work shall terminate
|
|
88
|
+
as of the date such litigation is filed.
|
|
89
|
+
|
|
90
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
91
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
92
|
+
modifications, and in Source or Object form, provided that You
|
|
93
|
+
meet the following conditions:
|
|
94
|
+
|
|
95
|
+
(a) You must give any other recipients of the Work or
|
|
96
|
+
Derivative Works a copy of this License; and
|
|
97
|
+
|
|
98
|
+
(b) You must cause any modified files to carry prominent notices
|
|
99
|
+
stating that You changed the files; and
|
|
100
|
+
|
|
101
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
102
|
+
that You distribute, all copyright, patent, trademark, and
|
|
103
|
+
attribution notices from the Source form of the Work,
|
|
104
|
+
excluding those notices that do not pertain to any part of
|
|
105
|
+
the Derivative Works; and
|
|
106
|
+
|
|
107
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
108
|
+
distribution, then any Derivative Works that You distribute must
|
|
109
|
+
include a readable copy of the attribution notices contained
|
|
110
|
+
within such NOTICE file, excluding those notices that do not
|
|
111
|
+
pertain to any part of the Derivative Works, in at least one
|
|
112
|
+
of the following places: within a NOTICE text file distributed
|
|
113
|
+
as part of the Derivative Works; within the Source form or
|
|
114
|
+
documentation, if provided along with the Derivative Works; or,
|
|
115
|
+
within a display generated by the Derivative Works, if and
|
|
116
|
+
wherever such third-party notices normally appear. The contents
|
|
117
|
+
of the NOTICE file are for informational purposes only and
|
|
118
|
+
do not modify the License. You may add Your own attribution
|
|
119
|
+
notices within Derivative Works that You distribute, alongside
|
|
120
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
121
|
+
that such additional attribution notices cannot be construed
|
|
122
|
+
as modifying the License.
|
|
123
|
+
|
|
124
|
+
You may add Your own copyright statement to Your modifications and
|
|
125
|
+
may provide additional or different license terms and conditions
|
|
126
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
127
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
128
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
129
|
+
the conditions stated in this License.
|
|
130
|
+
|
|
131
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
132
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
133
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
134
|
+
this License, without any additional terms or conditions.
|
|
135
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
136
|
+
the terms of any separate license agreement you may have executed
|
|
137
|
+
with Licensor regarding such Contributions.
|
|
138
|
+
|
|
139
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
140
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
141
|
+
except as required for reasonable and customary use in describing the
|
|
142
|
+
origin of the Work and reproducing the content of the NOTICE file.
|
|
143
|
+
|
|
144
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
145
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
146
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
147
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
148
|
+
implied, including, without limitation, any warranties or conditions
|
|
149
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
150
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
151
|
+
appropriateness of using or redistributing the Work and assume any
|
|
152
|
+
risks associated with Your exercise of permissions under this License.
|
|
153
|
+
|
|
154
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
155
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
156
|
+
unless required by applicable law (such as deliberate and grossly
|
|
157
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
158
|
+
liable to You for damages, including any direct, indirect, special,
|
|
159
|
+
incidental, or consequential damages of any character arising as a
|
|
160
|
+
result of this License or out of the use or inability to use the
|
|
161
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
162
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
163
|
+
other commercial damages or losses), even if such Contributor
|
|
164
|
+
has been advised of the possibility of such damages.
|
|
165
|
+
|
|
166
|
+
9. Accepting Warranty or Additional Liability. While redistributing
|
|
167
|
+
the Work or Derivative Works thereof, You may choose to offer,
|
|
168
|
+
and charge a fee for, acceptance of support, warranty, indemnity,
|
|
169
|
+
or other liability obligations and/or rights consistent with this
|
|
170
|
+
License. However, in accepting such obligations, You may act only
|
|
171
|
+
on Your own behalf and on Your sole responsibility, not on behalf
|
|
172
|
+
of any other Contributor, and only if You agree to indemnify,
|
|
173
|
+
defend, and hold each Contributor harmless for any liability
|
|
174
|
+
incurred by, or claims asserted against, such Contributor by reason
|
|
175
|
+
of your accepting any such warranty or additional liability.
|
|
176
|
+
|
|
177
|
+
END OF TERMS AND CONDITIONS
|
|
178
|
+
|
|
179
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
180
|
+
|
|
181
|
+
To apply the Apache License to your work, attach the following
|
|
182
|
+
boilerplate notice, with the fields enclosed by brackets "[]"
|
|
183
|
+
replaced with your own identifying information. (Don't include
|
|
184
|
+
the brackets!) The text should be enclosed in the appropriate
|
|
185
|
+
comment syntax for the file format. We also recommend that a
|
|
186
|
+
file or class name and description of purpose be included on the
|
|
187
|
+
same "printed page" as the copyright notice for easier
|
|
188
|
+
identification within third-party archives.
|
|
189
|
+
|
|
190
|
+
Copyright 2026 Avowa, Inc.
|
|
191
|
+
|
|
192
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
193
|
+
you may not use this file except in compliance with the License.
|
|
194
|
+
You may obtain a copy of the License at
|
|
195
|
+
|
|
196
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
197
|
+
|
|
198
|
+
Unless required by applicable law or agreed to in writing, software
|
|
199
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
200
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
201
|
+
See the License for the specific language governing permissions and
|
|
202
|
+
limitations under the License.
|
|
@@ -0,0 +1,358 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: avowa-runtime
|
|
3
|
+
Version: 0.11.0
|
|
4
|
+
Summary: Avowa Runtime SDK — emit live AI-inference signals to Avowa as evidence.
|
|
5
|
+
Author: Avowa, Inc.
|
|
6
|
+
License-Expression: Apache-2.0
|
|
7
|
+
Project-URL: Homepage, https://avowa.ai
|
|
8
|
+
Project-URL: Repository, https://github.com/Avowa-AI/avowa-runtime-sdk
|
|
9
|
+
Project-URL: Issues, https://github.com/Avowa-AI/avowa-runtime-sdk/issues
|
|
10
|
+
Keywords: avowa,ai-trust,evidence,observability,compliance
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: Programming Language :: Python :: 3
|
|
14
|
+
Classifier: Topic :: Security
|
|
15
|
+
Classifier: Topic :: System :: Monitoring
|
|
16
|
+
Requires-Python: >=3.9
|
|
17
|
+
Description-Content-Type: text/markdown
|
|
18
|
+
License-File: LICENSE
|
|
19
|
+
License-File: NOTICE
|
|
20
|
+
Dynamic: license-file
|
|
21
|
+
|
|
22
|
+
# Avowa Runtime SDK (Python)
|
|
23
|
+
|
|
24
|
+
Emit **live AI-inference signals** to Avowa so your AI-trust claims are backed by
|
|
25
|
+
runtime evidence — which model ran, in which region, whether customer data is used
|
|
26
|
+
for training, and guardrail outcomes. This is the evidence posture tools (Vanta/Drata)
|
|
27
|
+
can't capture: *how your AI actually behaves in production.*
|
|
28
|
+
|
|
29
|
+
- **Stdlib-only** — no dependencies added to your app.
|
|
30
|
+
- **Non-blocking** — a background worker sends signals; ~0ms added to inference.
|
|
31
|
+
|
|
32
|
+
## Install
|
|
33
|
+
|
|
34
|
+
**Fastest path — three lines, in the environment your app runs in** (Connections → Runtime SDK → *Generate setup token*):
|
|
35
|
+
|
|
36
|
+
```bash
|
|
37
|
+
pip install https://api.avowa.ai/public/sdk/avowa-runtime-sdk.zip # public, code only (PyPI follows)
|
|
38
|
+
avowa-run --init <setup-token> --api https://api.avowa.ai # writes avowa.env (0600); nothing to paste
|
|
39
|
+
avowa-run <your usual start command> # auto-instrumented, no code change
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
`avowa-run --doctor` then confirms the key, the secret and your clock without sending anything.
|
|
43
|
+
|
|
44
|
+
```bash
|
|
45
|
+
pip install ./sdk/python # local
|
|
46
|
+
# (not on PyPI yet — install from the zip your Avowa console hands you)
|
|
47
|
+
```
|
|
48
|
+
|
|
49
|
+
## Configure
|
|
50
|
+
Get the **ingest key** and **signing secret** from Avowa → **Connections → Avowa Runtime
|
|
51
|
+
SDK** (both are minted when you enable that connector). Then set env vars (or pass them to
|
|
52
|
+
the constructor):
|
|
53
|
+
|
|
54
|
+
```bash
|
|
55
|
+
export AVOWA_RUNTIME_KEY="..." # identifies your connection (from your avowa.env)
|
|
56
|
+
export AVOWA_RUNTIME_SECRET="avs_..." # signs each signal (tamper-evident)
|
|
57
|
+
export AVOWA_BASE_URL="https://api.your-avowa.com" # required — no default; https only
|
|
58
|
+
export AVOWA_REGION="us-east-1"
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
> **Tamper-evident and single-use (0.9+):** every request is HMAC-SHA256-signed over
|
|
62
|
+
> `"<unix-timestamp>.<nonce>.<body>"` (headers `X-Avowa-Timestamp` / `X-Avowa-Nonce` /
|
|
63
|
+
> `X-Avowa-Signature`). The server rejects bad signatures, requests outside a 5-minute window,
|
|
64
|
+
> and any nonce it has already seen — so a signal can't be forged, altered or replayed.
|
|
65
|
+
> **HTTPS only:** the SDK refuses a plain `http://` base URL unless it is localhost or a private
|
|
66
|
+
> network (override with `AVOWA_ALLOW_HTTP=1`), and sends nothing when no URL is set.
|
|
67
|
+
> **Off switch:** `AVOWA_DISABLE=1` — nothing is patched and nothing is sent, in every mode.
|
|
68
|
+
> Keep the secret server-side — never ship it to a browser.
|
|
69
|
+
|
|
70
|
+
## Usage
|
|
71
|
+
|
|
72
|
+
```python
|
|
73
|
+
from avowa_runtime import AvowaRuntime
|
|
74
|
+
avowa = AvowaRuntime(region="us-east-1") # reads AVOWA_RUNTIME_KEY / AVOWA_BASE_URL
|
|
75
|
+
|
|
76
|
+
# A) Wrap one client explicitly. Everything it records is OBSERVED (model, endpoint, latency,
|
|
77
|
+
# errors) and it runs the local injection/PII scans for you. Pass scan=False to disable them.
|
|
78
|
+
import anthropic, openai
|
|
79
|
+
client = avowa.instrument_anthropic(anthropic.Anthropic(), region="us-east-1")
|
|
80
|
+
client.messages.create(model="claude-opus-4-8", max_tokens=512, messages=[...])
|
|
81
|
+
|
|
82
|
+
client = avowa.instrument_openai(openai.OpenAI(), region="us-east-1")
|
|
83
|
+
client.chat.completions.create(model="gpt-4o", messages=[...])
|
|
84
|
+
|
|
85
|
+
# ...but prefer zero-code auto-instrumentation (next section) — it covers clients you can't
|
|
86
|
+
# reach by hand, which is where coverage is actually lost.
|
|
87
|
+
|
|
88
|
+
# B) Wrap a block (times it + records the model/region):
|
|
89
|
+
with avowa.track(model="claude-opus-4-8", region="us-east-1"):
|
|
90
|
+
resp = client.messages.create(...)
|
|
91
|
+
|
|
92
|
+
# C) Record explicitly (e.g. after a guardrail check):
|
|
93
|
+
avowa.capture(model="claude-opus-4-8", region="us-east-1",
|
|
94
|
+
train_on_data=False, guardrail_blocked=blocked, latency_ms=842)
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Each signal feeds the **runtime_sdk** connector. Run a capture/sync (or the Change Feed)
|
|
98
|
+
in Avowa and the signals become evidence. You only ever claim what you instrument — a field
|
|
99
|
+
you don't pass is not sent, so it proves nothing.
|
|
100
|
+
|
|
101
|
+
## Zero-code auto-instrumentation (recommended)
|
|
102
|
+
|
|
103
|
+
`instrument_openai(client)` asks you to find *where the client is constructed*. In a real
|
|
104
|
+
codebase that's the hard part — it may be a factory, five call sites, or inside LangChain /
|
|
105
|
+
LlamaIndex / CrewAI where you can't edit it at all. Anything you miss shows up as **low traffic**
|
|
106
|
+
rather than a missed integration, because Avowa publishes instrumentation coverage as a
|
|
107
|
+
denominator. Auto-instrumentation patches the client **classes**, so every instance is covered
|
|
108
|
+
no matter who builds it.
|
|
109
|
+
|
|
110
|
+
**Credentials first, and there is nothing to type.** Your SDK download contains `avowa.env`
|
|
111
|
+
already filled in — copy it into the directory you start your app from. The SDK reads **only
|
|
112
|
+
`AVOWA_*` keys** out of it and a real environment variable **always wins**, so Kubernetes Secrets
|
|
113
|
+
or Secrets Manager injection keeps working unchanged. (`AVOWA_ENV_FILE` points at a specific file;
|
|
114
|
+
`AVOWA_NO_ENV_FILE=1` switches it off.) Setting them by hand on macOS/Linux is `export VAR=value` —
|
|
115
|
+
plain `set VAR=value` is Windows CMD syntax and silently does nothing in bash/zsh.
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
# 1. Prefix how the process starts — nothing in your app changes. Best option for most teams.
|
|
119
|
+
avowa-run uvicorn app:main --host 0.0.0.0
|
|
120
|
+
avowa-run gunicorn -w 4 app:main
|
|
121
|
+
# Docker: CMD ["avowa-run", "uvicorn", "app:main"]
|
|
122
|
+
|
|
123
|
+
# 2. Can't change the start command? Install a start-up hook once per environment.
|
|
124
|
+
python -m avowa_runtime.auto --install # then set AVOWA_AUTO=1 (--uninstall to remove)
|
|
125
|
+
|
|
126
|
+
# 3. See what it patched.
|
|
127
|
+
python -m avowa_runtime.auto
|
|
128
|
+
```
|
|
129
|
+
|
|
130
|
+
```python
|
|
131
|
+
# 4. Or one import at your entry point (main.py, Django settings, Lambda handler):
|
|
132
|
+
import avowa_runtime.auto
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
All four read config from the environment only: `AVOWA_RUNTIME_KEY`, `AVOWA_RUNTIME_SECRET`,
|
|
136
|
+
`AVOWA_BASE_URL`, `AVOWA_REGION`, and `AVOWA_SCAN=0` to disable the local scan. Import order
|
|
137
|
+
doesn't matter — an import hook patches provider SDKs whenever they load. Gunicorn/uvicorn
|
|
138
|
+
workers inherit the environment, so workers are instrumented and Avowa sums their signals.
|
|
139
|
+
|
|
140
|
+
The `.pth` hook does nothing at all unless `AVOWA_AUTO=1`, so installing the SDK can never
|
|
141
|
+
silently start instrumenting something.
|
|
142
|
+
|
|
143
|
+
### Nothing showing up?
|
|
144
|
+
|
|
145
|
+
```bash
|
|
146
|
+
python -m avowa_runtime.auto --check # credentials + reachability. Sends NO signal:
|
|
147
|
+
# a synthetic event would show on your trust
|
|
148
|
+
# profile as inference that never happened.
|
|
149
|
+
AVOWA_DEBUG=1 avowa-run <your command> # trace patch → capture → send, metadata only
|
|
150
|
+
```
|
|
151
|
+
|
|
152
|
+
```
|
|
153
|
+
[avowa:debug] patched openai.OpenAI
|
|
154
|
+
[avowa:debug] captured gpt-4o-2026-05-13 via api.openai.com (32ms)
|
|
155
|
+
[avowa:debug] sent gpt-4o-2026-05-13 via api.openai.com (32ms) → HTTP 200
|
|
156
|
+
```
|
|
157
|
+
|
|
158
|
+
In order of likelihood: **no inference has run yet** (starting the app proves nothing — send a
|
|
159
|
+
request that calls the model); **the app doesn't use the `anthropic`/`openai` Python SDKs** (raw
|
|
160
|
+
`requests`/`httpx`, or a provider with no wrapper — use `capture()`); or **delivery is failing**.
|
|
161
|
+
The first failed send prints one line naming the cause — a wrong key, a wrong base URL, a blocked
|
|
162
|
+
host — and then stays quiet. Silence on that line means signals are being delivered.
|
|
163
|
+
|
|
164
|
+
`AVOWA_DEBUG` output is metadata-only, like the wire format. It never prints prompts or responses;
|
|
165
|
+
there's a test asserting that, because debug output is what people paste into tickets.
|
|
166
|
+
|
|
167
|
+
### What the SDK reads off each call
|
|
168
|
+
|
|
169
|
+
Controls that would otherwise be self-reported are visible in the call itself. The SDK reads the
|
|
170
|
+
facts and sends them as they are; Avowa evaluates them on its side and grades the result as
|
|
171
|
+
**observed (L3)** rather than asserted (L2). The SDK draws no conclusions of its own.
|
|
172
|
+
|
|
173
|
+
| Fact sent | Read from | Evidence it feeds |
|
|
174
|
+
|---|---|---|
|
|
175
|
+
| `requested_model` + `model` | the model id on the request, and the one in the response | AI supply-chain integrity |
|
|
176
|
+
| `stop_reason` | the provider's own stop / finish code | Guardrails / content filtering |
|
|
177
|
+
| `structured_output` | the call used `parse()` or supplied a schema | Output handling |
|
|
178
|
+
| `token_ceiling`, `error` | the request's output-token limit; the exception type on a failed call | Rate & cost limits |
|
|
179
|
+
| token usage (`input_tokens`, `output_tokens`, cache reads and writes) | the provider's own usage report on the response | Rate & cost limits · volume |
|
|
180
|
+
| `system_prompt_present`, `tool_choice`, `response_format_type`, `streaming` | the request's settings — whether a system prompt was set, never what it says | System-prompt protection · agency · output handling |
|
|
181
|
+
| `messages_user` / `_assistant` / `_tool`, `input_images` / `_files` / `_audio`, `prompt_chars`, `output_chars` | counts and sizes only | Autonomy · data types processed |
|
|
182
|
+
| `client_max_retries`, `client_timeout_s` | how your client is configured | AI-path resilience |
|
|
183
|
+
| `pii_families`, `injection_families` | which named pattern families the local scan matched (never the text) | Sensitive-data leakage · prompt injection |
|
|
184
|
+
| `temperature`, `top_p`, `seed`, `choices_n`, `stop_sequences` (a count), `reasoning_effort`, `thinking_enabled` | the request's sampling and reasoning settings | Reproducibility · output handling |
|
|
185
|
+
| `store_requested`, `service_tier_requested` / `service_tier` | whether the app asked the provider to store the output; the tier requested and served | Data retention · no-training |
|
|
186
|
+
| `end_user_id_present`, `metadata_present` | **whether** an end-user id or metadata travelled with the call — never the id | Accountability · per-user attribution |
|
|
187
|
+
| `provider_filtered`, `provider_filter_categories`, `provider_block_reason`, `guardrail_configured`, `safety_settings` (a count) | the provider's **own** safety verdict (Azure content filters, Gemini block reasons, a Bedrock guardrail) — category names only | Guardrails / content filtering |
|
|
188
|
+
| `system_fingerprint` | the provider's backend configuration id | AI supply-chain integrity · model change |
|
|
189
|
+
| `http_status`, `provider_error_code`, `retry_after_s` | a failed call's status and the provider's error **code** — never the error message, which can echo the request | Rate & cost limits · AI-path resilience |
|
|
190
|
+
| `ttft_ms` | time to the first streamed chunk | Availability · performance |
|
|
191
|
+
| `provider_lib` / `provider_lib_version`, `python_version`, `app_frameworks` | which provider library, Python and agent frameworks the app runs | AI bill of materials |
|
|
192
|
+
| `app_region` + `app_region_source` | the region code from your platform's own variable (`AWS_REGION`, `FLY_REGION` …) | Where processing happens — cross-border transfers |
|
|
193
|
+
|
|
194
|
+
### New metadata without a redeploy
|
|
195
|
+
|
|
196
|
+
Providers add metadata fields over time (a new safety verdict, a new service tier). Rather than ask
|
|
197
|
+
you to upgrade for each one, Avowa can ask the SDK to read a further **metadata** field by name. The
|
|
198
|
+
list arrives on the response to a signal the SDK already sends, authenticated with your connection's
|
|
199
|
+
secret, and only when your copy is out of date.
|
|
200
|
+
|
|
201
|
+
- **Data, never code.** An entry is a name, a provider and an attribute path. The SDK reads that path
|
|
202
|
+
and nothing else; no code is downloaded or run.
|
|
203
|
+
- **The guard is in the SDK, not in the list.** A path through any content-bearing name (`content`,
|
|
204
|
+
`text`, `messages`, `input`, `output`, `arguments`, `system` …) or a private attribute is refused,
|
|
205
|
+
and only a number, true/false or a short code (letters, digits, `_.:-`, at most 40 characters) can
|
|
206
|
+
come back. No list Avowa sends can widen what leaves your process.
|
|
207
|
+
- **See it, or switch it off.** `python -m avowa_runtime.auto` prints the active list.
|
|
208
|
+
`AVOWA_REMOTE_FIELDS=off` turns it off; only the built-in metadata above is then read.
|
|
209
|
+
|
|
210
|
+
**If your app gives the model tools, the agentic evidence comes free too.** Nobody else can produce
|
|
211
|
+
this, because it isn't in a log or a config file — it only exists in the call:
|
|
212
|
+
|
|
213
|
+
| Flag | How it is derived | Claim it proves |
|
|
214
|
+
|---|---|---|
|
|
215
|
+
| `tools_offered` | the tool definitions supplied on this call — the grant, as it really was | Agent tool inventory |
|
|
216
|
+
| `tools_invoked` | the tool-use blocks the model actually returned (parallel calls included) | Agent tool inventory · tool-permission scoping |
|
|
217
|
+
| `autonomy_depth` | prior assistant turns that requested tools — steps taken before returning to a human | Autonomy bounds |
|
|
218
|
+
|
|
219
|
+
**Tool names only.** Schemas, descriptions and arguments are read for the name and dropped
|
|
220
|
+
in-process: they carry business logic and sometimes examples containing real data, so they are
|
|
221
|
+
treated exactly like prompt text.
|
|
222
|
+
|
|
223
|
+
Unused grants are the point. A tool handed to the model on every call and never invoked is standing
|
|
224
|
+
authority nobody is watching — OWASP LLM06 — and Avowa reports it as a finding. Scoping is only
|
|
225
|
+
*proven* when every granted tool is exercised; an unused one is never called proof of over-scoping
|
|
226
|
+
either, because a short window can't tell a rarely-used tool from an unnecessary one.
|
|
227
|
+
|
|
228
|
+
Nothing here defaults to `True`, and a flag is **omitted entirely** when the call does not answer
|
|
229
|
+
the question — a field the SDK cannot see is never filled in on your behalf. `autonomy_depth` is
|
|
230
|
+
reported as unknown rather than `0` when the message history isn't a shape it recognises, because
|
|
231
|
+
`0` would read as "fully attended".
|
|
232
|
+
|
|
233
|
+
Everything else (`human_reviewed`, `decision_logged`, `ai_disclosed`, `grounded`,
|
|
234
|
+
`rationale_present`, `fairness_checked`, `uncertainty_flagged`, `rate_limited`,
|
|
235
|
+
`system_prompt_protected`, `eval_score`) is genuinely invisible from the call, so it stays an
|
|
236
|
+
explicit `capture(...)` argument and is graded self-reported. **Pass only what your app actually
|
|
237
|
+
enforces** — asserting a control you don't have is the thing Avowa exists to catch.
|
|
238
|
+
|
|
239
|
+
Where a signal is built from ONE field, calls the SDK observed are graded on their own and the
|
|
240
|
+
app-reported ones are named and excluded — an assertion can neither inflate an observation nor
|
|
241
|
+
suppress it. Where a signal blends several fields (guardrails combines your guardrail names and PII
|
|
242
|
+
counts with the block outcome) it stays as strong as its weakest field, so every call must be
|
|
243
|
+
observed for it to be graded observed.
|
|
244
|
+
|
|
245
|
+
`action_class` (read / write / external / financial) is **not** derived. The SDK sees a tool's name,
|
|
246
|
+
not its consequence, and guessing would put a judgement in your mouth — pass it explicitly if you
|
|
247
|
+
want human-approval coverage measured over consequential actions only.
|
|
248
|
+
|
|
249
|
+
### Which providers auto-instrument
|
|
250
|
+
|
|
251
|
+
| Provider | Classes patched | Notes |
|
|
252
|
+
|---|---|---|
|
|
253
|
+
| Anthropic | `Anthropic`, `AsyncAnthropic` | covers `messages.create` and `messages.parse` |
|
|
254
|
+
| Claude on Bedrock / Vertex | `AnthropicBedrock`, `AnthropicVertex` | covered when you use the `anthropic` SDK's own clients |
|
|
255
|
+
| OpenAI | `OpenAI`, `AsyncOpenAI` | covers `chat.completions.create` / `.parse` (incl. the `beta` namespace) and `responses.create` / `.parse` |
|
|
256
|
+
| Azure OpenAI | `AzureOpenAI`, `AsyncAzureOpenAI` | the Azure host is recorded, which is what evidences the inference region instead of assuming it |
|
|
257
|
+
| Any OpenAI-compatible endpoint (Together, Groq, Fireworks, vLLM, OpenRouter, local) | via the `openai` classes | the real host is recorded, not `api.openai.com` |
|
|
258
|
+
| Amazon Bedrock (boto3 `bedrock-runtime`) | `from avowa_runtime.adapters import instrument_bedrock` → `instrument_bedrock(avowa, client)` | `converse` and `invoke_model`; the Bedrock endpoint host and region, `stopReason` (a `guardrail_intervened` stop is the guardrail firing, observed), tool grants from `toolConfig` |
|
|
259
|
+
| Google Gemini (`google-genai`) | `instrument_gemini(avowa, client)` | `models.generate_content` (sync and `aio`); served `model_version`, `finish_reason` (SAFETY / PROHIBITED_CONTENT = blocked), Vertex vs AI Studio host |
|
|
260
|
+
| LangChain / LangGraph | `from avowa_runtime.langchain import AvowaCallbackHandler` → `callbacks=[AvowaCallbackHandler()]` | one signal per model call from the framework's own events, tool names from `on_tool_start`. Do not combine with a wrapped client — the call would count twice |
|
|
261
|
+
| `mistralai`, `cohere` | — | no wrapper yet: `avowa.track(...)` / `avowa.capture(...)`, graded as self-reported |
|
|
262
|
+
|
|
263
|
+
**Async clients are fully supported** — the signal is recorded after the `await`, not when the
|
|
264
|
+
coroutine is created, so `latency_ms` is the real call duration and a failure is recorded as a
|
|
265
|
+
failure.
|
|
266
|
+
|
|
267
|
+
**Streaming is captured when the stream ends.** With `stream=True` the wrapper returns a proxy
|
|
268
|
+
over the provider's iterator (sync or async, context manager included) that watches the chunks go
|
|
269
|
+
by — the served model, the stop/finish reason, the output text for the local scan — and records
|
|
270
|
+
once, with latency measured to the last chunk, when the stream is exhausted or closed. Nothing is
|
|
271
|
+
recorded before that, because nothing is known before that. Anthropic's `messages.stream()` helper
|
|
272
|
+
is not wrapped yet; call `capture(...)` after it closes.
|
|
273
|
+
|
|
274
|
+
**Check the install without sending anything:** `python -m avowa_runtime.auto --doctor` asks the
|
|
275
|
+
server who this key is — connection, environment, whether the plan allows capture, whether your
|
|
276
|
+
secret signs correctly, your clock skew, and when it last heard from you.
|
|
277
|
+
|
|
278
|
+
Nothing about your provider changes how evidence is graded. What matters is observed vs
|
|
279
|
+
asserted, below — a manual `capture()` with observed fields outranks a wrapped call carrying
|
|
280
|
+
only self-reported booleans.
|
|
281
|
+
|
|
282
|
+
## Observed vs asserted — how Avowa grades what you send
|
|
283
|
+
|
|
284
|
+
Not every field is worth the same, and Avowa will not pretend otherwise.
|
|
285
|
+
|
|
286
|
+
| | What it is | Grade |
|
|
287
|
+
|---|---|---|
|
|
288
|
+
| **Observed** | The SDK worked it out itself — the model the API returned, the host it dialled, latency, errors, and the counts its own local detectors produced. | **L3 · live-verified** |
|
|
289
|
+
| **Asserted** | A boolean your application passed about itself (`output_validated=True`). The HMAC proves the transport wasn't tampered with; it proves nothing about whether the claim is true. | **L2 · runtime-asserted** |
|
|
290
|
+
|
|
291
|
+
An assertion is still worth more than a questionnaire answer — it's continuous, timestamped and
|
|
292
|
+
signed — but it is a self-report, and a buyer is told so. **Prefer the observed fields.** They cost
|
|
293
|
+
you nothing extra and they're the only ones that read as independently verified.
|
|
294
|
+
|
|
295
|
+
Avowa also reports **instrumentation coverage**: if a flag rides on 5 of your 1,000 calls, the
|
|
296
|
+
evidence says `5/1000 (0.5% of traffic)`, not `5/5`.
|
|
297
|
+
|
|
298
|
+
**Core**
|
|
299
|
+
|
|
300
|
+
| Field | Proves | Graded |
|
|
301
|
+
|---|---|---|
|
|
302
|
+
| `model` | `ai_components.models` | observed |
|
|
303
|
+
| `endpoint` | `subprocessors` (observed egress) + `data.data_residency_jurisdiction` where the host encodes a region | observed |
|
|
304
|
+
| `scan_prompt` → `injection_suspected` | `ai_security.prompt_injection` | observed |
|
|
305
|
+
| `scan_output` → `pii_detected` | `ai_security.sensitive_disclosure` | observed |
|
|
306
|
+
| `error` / `retries` / `fallback_model` | `resilience.ai_path` | observed |
|
|
307
|
+
| `region` | `data.data_residency_jurisdiction` | observed if derivable from `endpoint`, else asserted |
|
|
308
|
+
| `train_on_data=false` | `training_posture.trains_model_weights_on_customer_data` | asserted |
|
|
309
|
+
| `guardrail_blocked` / `guardrail` / `blocked_category` | `responsible_ai.model_monitoring` + `guardrails` (+ live red-teaming) | asserted |
|
|
310
|
+
|
|
311
|
+
### Local detection, metadata egress
|
|
312
|
+
|
|
313
|
+
`scan_prompt` and `scan_output` are matched **inside your process** and dropped. Avowa receives an
|
|
314
|
+
integer count and nothing else — never the text. That is what lets a leak or an attack be evidenced
|
|
315
|
+
without Avowa ever processing your prompts.
|
|
316
|
+
|
|
317
|
+
A detection is evidence an attack *arrived*, not that it was *stopped*: injection patterns your app
|
|
318
|
+
doesn't report blocking, or PII still present in egress after your redaction ran, are reported as
|
|
319
|
+
**unresolved exposure** rather than a passing control.
|
|
320
|
+
|
|
321
|
+
**Responsible AI** (pass the flags your app enforces)
|
|
322
|
+
|
|
323
|
+
| Field | Proves |
|
|
324
|
+
|---|---|
|
|
325
|
+
| `decision_logged` | `responsible_ai.decision_audit` + `prompt_output_logging` |
|
|
326
|
+
| `human_reviewed` | `ai_components.human_oversight` |
|
|
327
|
+
| `rationale_present` | `responsible_ai.decision_logic` (explainability) |
|
|
328
|
+
| `ai_disclosed` | `responsible_ai.ai_disclosure` |
|
|
329
|
+
| `uncertainty_flagged` | `responsible_ai.uncertainty_handling` |
|
|
330
|
+
| `fairness_checked` | `responsible_ai.bias_detection` |
|
|
331
|
+
|
|
332
|
+
**AI security — OWASP LLM Top 10 + agentic** (the moat: proven live from the call path, which no posture tool or static pentest can show)
|
|
333
|
+
|
|
334
|
+
| Field | Proves | OWASP |
|
|
335
|
+
|---|---|---|
|
|
336
|
+
| `injection_blocked` | `ai_security.prompt_injection` | LLM01 |
|
|
337
|
+
| `pii_redacted` | `ai_security.sensitive_disclosure` | LLM02 |
|
|
338
|
+
| `model_pinned` | `ai_security.model_supply_chain` | LLM03 |
|
|
339
|
+
| `output_validated` | `ai_security.output_handling` | LLM05 |
|
|
340
|
+
| `tool_authorized` | `ai_security.excessive_agency` (agent tool-scoping) | LLM06 |
|
|
341
|
+
| `action_authorized` | `ai_security.action_authorization` (high-impact action gating) | agentic |
|
|
342
|
+
| `system_prompt_protected` | `ai_security.system_prompt_protection` | LLM07 |
|
|
343
|
+
| `grounded` | `ai_security.groundedness` (hallucination resistance) | LLM09 |
|
|
344
|
+
| `rate_limited` | `ai_security.rate_limiting` (abuse / DoS) | LLM10 |
|
|
345
|
+
| `eval_score` (number) | `ai_governance.model_evaluation` | — |
|
|
346
|
+
|
|
347
|
+
```python
|
|
348
|
+
# prove AI-security controls live — pass what your inference path enforces on each call:
|
|
349
|
+
avowa.capture(model="claude-opus-4-8", region="us-east-1", train_on_data=False,
|
|
350
|
+
injection_blocked=False, pii_redacted=True, output_validated=True,
|
|
351
|
+
system_prompt_protected=True, grounded=True, tool_authorized=True,
|
|
352
|
+
action_authorized=True, model_pinned=True, rate_limited=False, eval_score=0.94)
|
|
353
|
+
```
|
|
354
|
+
|
|
355
|
+
## Notes
|
|
356
|
+
- Sends are best-effort and never raise into your request path.
|
|
357
|
+
- `avowa.flush()` is registered at process exit; call it manually before a short-lived job ends.
|
|
358
|
+
- Only metadata is sent — never prompts, outputs, or customer data.
|