@jxburros/ai-nugget 0.4.1 → 0.7.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (187) hide show
  1. package/README.md +158 -382
  2. package/UPGRADING.md +182 -0
  3. package/dist/adapters/engines/anthropic.d.ts +7 -1
  4. package/dist/adapters/engines/anthropic.js +82 -37
  5. package/dist/adapters/engines/base.d.ts +29 -2
  6. package/dist/adapters/engines/base.js +52 -8
  7. package/dist/adapters/engines/google.d.ts +7 -1
  8. package/dist/adapters/engines/google.js +80 -29
  9. package/dist/adapters/engines/ollama.d.ts +2 -1
  10. package/dist/adapters/engines/ollama.js +57 -15
  11. package/dist/adapters/engines/openaiChat.d.ts +8 -1
  12. package/dist/adapters/engines/openaiChat.js +109 -29
  13. package/dist/adapters/profiles.d.ts +21 -1
  14. package/dist/adapters/profiles.js +12 -2
  15. package/dist/agent/loop.d.ts +45 -1
  16. package/dist/agent/loop.js +117 -12
  17. package/dist/cjs/adapters/engines/anthropic.js +326 -0
  18. package/dist/cjs/adapters/engines/base.js +147 -0
  19. package/dist/cjs/adapters/engines/google.js +291 -0
  20. package/dist/cjs/adapters/engines/ollama.js +205 -0
  21. package/dist/cjs/adapters/engines/openaiChat.js +312 -0
  22. package/dist/cjs/adapters/index.js +22 -0
  23. package/dist/cjs/adapters/profiles.js +125 -0
  24. package/dist/cjs/agent/index.js +18 -0
  25. package/dist/cjs/agent/loop.js +373 -0
  26. package/dist/cjs/agent/tools.js +51 -0
  27. package/dist/cjs/connect.js +27 -0
  28. package/dist/cjs/errors.js +110 -0
  29. package/dist/cjs/handler.js +586 -0
  30. package/dist/cjs/index.js +28 -0
  31. package/dist/cjs/json.js +121 -0
  32. package/dist/cjs/keys.js +81 -0
  33. package/dist/cjs/package.json +3 -0
  34. package/dist/cjs/policy.js +42 -0
  35. package/dist/cjs/reasoning.js +159 -0
  36. package/dist/cjs/redact.js +78 -0
  37. package/dist/cjs/tokens.js +22 -0
  38. package/dist/cjs/transport.js +172 -0
  39. package/dist/cjs/types.js +6 -0
  40. package/dist/cjs/util.js +126 -0
  41. package/dist/errors.js +25 -1
  42. package/dist/handler.d.ts +67 -1
  43. package/dist/handler.js +266 -61
  44. package/dist/index.d.ts +1 -0
  45. package/dist/index.js +1 -0
  46. package/dist/reasoning.d.ts +45 -0
  47. package/dist/reasoning.js +154 -0
  48. package/dist/redact.d.ts +18 -0
  49. package/dist/redact.js +29 -1
  50. package/dist/transport.d.ts +3 -1
  51. package/dist/transport.js +21 -1
  52. package/dist/types.d.ts +112 -2
  53. package/dist/util.d.ts +22 -0
  54. package/dist/util.js +59 -0
  55. package/nugget/VERSION.txt +2 -2
  56. package/nugget/dist/adapters/engines/anthropic.d.ts +14 -0
  57. package/nugget/dist/adapters/engines/anthropic.d.ts.map +1 -0
  58. package/nugget/dist/adapters/engines/anthropic.js +323 -0
  59. package/nugget/dist/adapters/engines/anthropic.js.map +1 -0
  60. package/nugget/dist/adapters/engines/base.d.ts +50 -0
  61. package/nugget/dist/adapters/engines/base.d.ts.map +1 -0
  62. package/nugget/dist/adapters/engines/base.js +137 -0
  63. package/nugget/dist/adapters/engines/base.js.map +1 -0
  64. package/nugget/dist/adapters/engines/google.d.ts +24 -0
  65. package/nugget/dist/adapters/engines/google.d.ts.map +1 -0
  66. package/nugget/dist/adapters/engines/google.js +288 -0
  67. package/nugget/dist/adapters/engines/google.js.map +1 -0
  68. package/nugget/dist/adapters/engines/ollama.d.ts +11 -0
  69. package/nugget/dist/adapters/engines/ollama.d.ts.map +1 -0
  70. package/nugget/dist/adapters/engines/ollama.js +202 -0
  71. package/nugget/dist/adapters/engines/ollama.js.map +1 -0
  72. package/nugget/dist/adapters/engines/openaiChat.d.ts +22 -0
  73. package/nugget/dist/adapters/engines/openaiChat.d.ts.map +1 -0
  74. package/nugget/dist/adapters/engines/openaiChat.js +308 -0
  75. package/nugget/dist/adapters/engines/openaiChat.js.map +1 -0
  76. package/nugget/dist/adapters/index.d.ts +5 -0
  77. package/nugget/dist/adapters/index.d.ts.map +1 -0
  78. package/nugget/dist/adapters/index.js +17 -0
  79. package/nugget/dist/adapters/index.js.map +1 -0
  80. package/nugget/dist/adapters/profiles.d.ts +62 -0
  81. package/nugget/dist/adapters/profiles.d.ts.map +1 -0
  82. package/nugget/dist/adapters/profiles.js +121 -0
  83. package/nugget/dist/adapters/profiles.js.map +1 -0
  84. package/nugget/dist/agent/index.d.ts +3 -0
  85. package/nugget/dist/agent/index.d.ts.map +1 -0
  86. package/nugget/dist/agent/index.js +3 -0
  87. package/nugget/dist/agent/index.js.map +1 -0
  88. package/nugget/dist/agent/loop.d.ts +114 -0
  89. package/nugget/dist/agent/loop.d.ts.map +1 -0
  90. package/nugget/dist/agent/loop.js +371 -0
  91. package/nugget/dist/agent/loop.js.map +1 -0
  92. package/nugget/dist/agent/tools.d.ts +29 -0
  93. package/nugget/dist/agent/tools.d.ts.map +1 -0
  94. package/nugget/dist/agent/tools.js +48 -0
  95. package/nugget/dist/agent/tools.js.map +1 -0
  96. package/nugget/dist/cjs/adapters/engines/anthropic.js +326 -0
  97. package/nugget/dist/cjs/adapters/engines/base.js +147 -0
  98. package/nugget/dist/cjs/adapters/engines/google.js +291 -0
  99. package/nugget/dist/cjs/adapters/engines/ollama.js +205 -0
  100. package/nugget/dist/cjs/adapters/engines/openaiChat.js +312 -0
  101. package/nugget/dist/cjs/adapters/index.js +22 -0
  102. package/nugget/dist/cjs/adapters/profiles.js +125 -0
  103. package/nugget/dist/cjs/agent/index.js +18 -0
  104. package/nugget/dist/cjs/agent/loop.js +373 -0
  105. package/nugget/dist/cjs/agent/tools.js +51 -0
  106. package/nugget/dist/cjs/connect.js +27 -0
  107. package/nugget/dist/cjs/errors.js +110 -0
  108. package/nugget/dist/cjs/handler.js +586 -0
  109. package/nugget/dist/cjs/index.js +28 -0
  110. package/nugget/dist/cjs/json.js +121 -0
  111. package/nugget/dist/cjs/keys.js +81 -0
  112. package/nugget/dist/cjs/package.json +3 -0
  113. package/nugget/dist/cjs/policy.js +42 -0
  114. package/nugget/dist/cjs/reasoning.js +159 -0
  115. package/nugget/dist/cjs/redact.js +78 -0
  116. package/nugget/dist/cjs/tokens.js +22 -0
  117. package/nugget/dist/cjs/transport.js +172 -0
  118. package/nugget/dist/cjs/types.js +6 -0
  119. package/nugget/dist/cjs/util.js +126 -0
  120. package/nugget/dist/connect.d.ts +31 -0
  121. package/nugget/dist/connect.d.ts.map +1 -0
  122. package/nugget/dist/connect.js +25 -0
  123. package/nugget/dist/connect.js.map +1 -0
  124. package/nugget/dist/errors.d.ts +22 -0
  125. package/nugget/dist/errors.d.ts.map +1 -0
  126. package/nugget/dist/errors.js +104 -0
  127. package/nugget/dist/errors.js.map +1 -0
  128. package/nugget/dist/handler.d.ts +114 -0
  129. package/nugget/dist/handler.d.ts.map +1 -0
  130. package/nugget/dist/handler.js +582 -0
  131. package/nugget/dist/handler.js.map +1 -0
  132. package/nugget/dist/index.d.ts +13 -0
  133. package/nugget/dist/index.d.ts.map +1 -0
  134. package/nugget/dist/index.js +13 -0
  135. package/nugget/dist/index.js.map +1 -0
  136. package/nugget/dist/json.d.ts +7 -0
  137. package/nugget/dist/json.d.ts.map +1 -0
  138. package/nugget/dist/json.js +114 -0
  139. package/nugget/dist/json.js.map +1 -0
  140. package/nugget/dist/keys.d.ts +7 -0
  141. package/nugget/dist/keys.d.ts.map +1 -0
  142. package/nugget/dist/keys.js +75 -0
  143. package/nugget/dist/keys.js.map +1 -0
  144. package/nugget/dist/policy.d.ts +6 -0
  145. package/nugget/dist/policy.d.ts.map +1 -0
  146. package/nugget/dist/policy.js +37 -0
  147. package/nugget/dist/policy.js.map +1 -0
  148. package/nugget/dist/reasoning.d.ts +45 -0
  149. package/nugget/dist/reasoning.d.ts.map +1 -0
  150. package/nugget/dist/reasoning.js +154 -0
  151. package/nugget/dist/reasoning.js.map +1 -0
  152. package/nugget/dist/redact.d.ts +26 -0
  153. package/nugget/dist/redact.d.ts.map +1 -0
  154. package/nugget/dist/redact.js +74 -0
  155. package/nugget/dist/redact.js.map +1 -0
  156. package/nugget/dist/tokens.d.ts +5 -0
  157. package/nugget/dist/tokens.d.ts.map +1 -0
  158. package/nugget/dist/tokens.js +18 -0
  159. package/nugget/dist/tokens.js.map +1 -0
  160. package/nugget/dist/transport.d.ts +30 -0
  161. package/nugget/dist/transport.d.ts.map +1 -0
  162. package/nugget/dist/transport.js +165 -0
  163. package/nugget/dist/transport.js.map +1 -0
  164. package/nugget/dist/types.d.ts +301 -0
  165. package/nugget/dist/types.d.ts.map +1 -0
  166. package/nugget/dist/types.js +4 -0
  167. package/nugget/dist/types.js.map +1 -0
  168. package/nugget/dist/util.d.ts +31 -0
  169. package/nugget/dist/util.d.ts.map +1 -0
  170. package/nugget/dist/util.js +114 -0
  171. package/nugget/dist/util.js.map +1 -0
  172. package/nugget/src/adapters/engines/anthropic.ts +78 -37
  173. package/nugget/src/adapters/engines/base.ts +50 -7
  174. package/nugget/src/adapters/engines/google.ts +79 -29
  175. package/nugget/src/adapters/engines/ollama.ts +56 -17
  176. package/nugget/src/adapters/engines/openaiChat.ts +104 -30
  177. package/nugget/src/adapters/profiles.ts +53 -3
  178. package/nugget/src/agent/loop.ts +156 -13
  179. package/nugget/src/errors.ts +25 -1
  180. package/nugget/src/handler.ts +308 -59
  181. package/nugget/src/index.ts +1 -0
  182. package/nugget/src/reasoning.ts +174 -0
  183. package/nugget/src/redact.ts +32 -1
  184. package/nugget/src/transport.ts +20 -1
  185. package/nugget/src/types.ts +113 -1
  186. package/nugget/src/util.ts +64 -0
  187. package/package.json +18 -5
package/README.md CHANGED
@@ -1,432 +1,208 @@
1
1
  # AI Nugget
2
2
 
3
- `AI Nugget` is a small, zero-dependency, isomorphic TypeScript nugget for
4
- talking to AI model providers through one pipeline:
3
+ **Version 0.7.0** · [What's changed since 0.6.0](#whats-changed-since-060) · [Changelog](./CHANGELOG.md) · [Upgrading](./UPGRADING.md)
4
+
5
+ A small, zero-dependency, isomorphic TypeScript nugget for talking to AI model
6
+ providers through one pipeline:
5
7
 
6
8
  ```
7
9
  policy → key resolution → beforeCall hook → concurrency/retry → provider adapter → redacted telemetry
8
10
  ```
9
11
 
10
- It runs identically in Node 20.19+/22.12+, the browser main thread, and Web Workers using
11
- only `fetch`, `ReadableStream`, `AbortController`, and `TextDecoder`. It is
12
- intentionally **not** a router, prompt library, memory system, or secrets vault:
13
- apps choose models, own prompts, store secrets, and decide what policy to enforce.
14
- The package is MIT licensed.
12
+ 19 provider profiles over 4 protocol engines. Runs identically in Node, the
13
+ browser main thread, and Web Workers using only `fetch`, `ReadableStream`,
14
+ `AbortController`, and `TextDecoder`. MIT licensed.
15
+
16
+ It is intentionally **not** a router, prompt library, memory system, or secrets
17
+ vault: apps choose models, own prompts, store secrets, and decide policy.
18
+
19
+ ## Quick start
15
20
 
16
- See `design.md` for the full contract. The original evidence base, phased
17
- build/adoption plan, and dev handoff are archived under `docs/archive/`.
21
+ Requires Node `^20.19.0` or `>=22.12.0` ([why](./docs/distribution.md#requirements)).
18
22
 
19
- ## Install / use
23
+ ```bash
24
+ npm install @jxburros/ai-nugget
25
+ ```
20
26
 
21
27
  ```ts
22
28
  import { AIHandler, envKeySource } from '@jxburros/ai-nugget';
23
29
 
24
30
  const handler = new AIHandler({ keySource: envKeySource() });
25
31
 
32
+ const conn = { id: 'main', provider: 'openai', keyRef: { kind: 'env', name: 'OPENAI_API_KEY' } };
33
+ const req = { model: 'gpt-4o-mini', messages: [{ role: 'user', content: 'Hello!' }] };
34
+
26
35
  // Non-streaming
27
- const result = await handler.chat(
28
- { id: 'main', provider: 'openai', keyRef: { kind: 'env', name: 'OPENAI_API_KEY' } },
29
- { model: 'gpt-4o-mini', messages: [{ role: 'user', content: 'Hello!' }] },
30
- );
36
+ const result = await handler.chat(conn, req);
31
37
  console.log(result.text, result.usage, result.finishReason);
32
38
 
33
- // Streaming (works in Node route handlers, React, and Web Workers)
39
+ // Streaming (Node route handlers, React, Web Workers — same code)
34
40
  for await (const event of handler.stream(conn, req)) {
35
41
  if (event.type === 'delta') process.stdout.write(event.text);
36
42
  if (event.type === 'done') console.log('\n', event.result.usage);
43
+ if (event.type === 'error') console.error(event.error.kind, event.error.message);
37
44
  }
38
45
  ```
39
46
 
40
- Repos that cannot take a package dependency can vendor the generated `nugget/`
41
- folder (see below).
42
-
43
- ## Providers
44
-
45
- Four **protocol engines** own the wire formats; a data-driven **profile table**
46
- turns each named provider into defaults + quirks on top of an engine, so adding
47
- an OpenAI-compatible provider is a table row, not another copy of the SSE loop.
48
-
49
- | Engine | Providers | Streaming | Tools | JSON mode |
50
- |---|---|---|---|---|
51
- | `openaiChat` | `openai`, `azure-openai`, `openrouter`, `groq`, `deepseek`, `mistral`, `together`, `fireworks`, `lmstudio`, `llamacpp`, `vllm`, `openai-compat` | SSE + `stream_options.include_usage` | `tools`/`tool_calls` deltas | `response_format` (`json_schema` where supported, otherwise `json_object`) |
52
- | `anthropic` | `anthropic` | SSE (`content_block_delta`) | `tool_use` blocks (streamed) | forced-tool JSON mode |
53
- | `google` | `google` | SSE (`streamGenerateContent`) | `functionDeclarations` | `responseMimeType` |
54
- | `ollama` | `ollama` | NDJSON | native `tools` | `format: json` |
55
-
56
- `adapterFor(provider)` lives once, in `adapters/index.ts`. Unknown provider keys
57
- with a `baseUrl` resolve to the `openai-compat` escape hatch. **Grok/xAI is not
58
- blocked and is never officially integrated** — point `openai-compat` at any
59
- endpoint if you must; that is your configuration, not a supported profile.
60
-
61
- **Wire URL for `openai-compat` and the other local runtimes.** `openai-compat`,
62
- `lmstudio`, `llamacpp`, and `vllm` have no `urlTemplate` in the profile table,
63
- so the `openaiChat` engine falls back to `POST {baseUrl}/chat/completions`
64
- (no `/v1` prefix — include it in `baseUrl` yourself if your server needs it,
65
- e.g. `http://localhost:8080/v1`). This matters most when standing up a local
66
- mock or test server: it has to listen on that exact path.
67
-
68
- Model identity is always `(source, model)` — `modelRef(source, model)` gives the
69
- canonical `provider/model` key so the same weights served by different hosts stay
70
- distinct and comparable.
71
-
72
- ### Provider capabilities
73
-
74
- Beyond wire-format `quirks` (auth, URL templates, whether a stream reports
75
- usage), each profile also carries `capabilities` — the higher-level questions
76
- apps need answered before they pick behavior:
47
+ No key? Point at a local model instead — same code, no `keyRef`:
77
48
 
78
49
  ```ts
79
- interface ProviderCapabilities {
80
- nativeTools: boolean; // reliable native tool/function-calling for this
81
- // provider's protocol (not just model-dependent)
82
- jsonMode: boolean; // a provider-enforced structured-output mode exists
83
- local: boolean; // runs on localhost / user-controlled infra
84
- embeddable: boolean; // meant to run as a local sidecar, not a hosted service
85
- }
50
+ const conn = { id: 'local', provider: 'ollama', baseUrl: 'http://127.0.0.1:11434' };
86
51
  ```
87
52
 
88
- Hosted cloud providers (`openai`, `azure-openai`, `openrouter`, `groq`,
89
- `deepseek`, `mistral`, `together`, `fireworks`, `anthropic`, `google`) ship
90
- `{ nativeTools: true, jsonMode: true, local: false, embeddable: false }`. Local
91
- runtimes (`lmstudio`, `llamacpp`, `vllm`, and the `openai-compat` escape hatch)
92
- ship the conservative `{ nativeTools: false, jsonMode: false, local: true,
93
- embeddable: true }`, since native tool support there is model-dependent, not
94
- protocol-guaranteed. `ollama` is the one exception with `jsonMode: true` —
95
- its `format: 'json'` is engine-enforced regardless of which model is loaded.
96
- These are per-provider defaults, not per-model guarantees: a caller that knows
97
- its specific loaded model better should configure behavior explicitly (e.g.
98
- `toolMode: 'native'`) rather than rely on the default.
99
-
100
- Access via `profileFor(provider, baseUrl).capabilities`.
101
-
102
- **These are provider-level defaults, not per-model guarantees.** A specific
103
- model behind a "hosted" provider can still lack reliable native tool-calling
104
- (and a specific local model behind a "local" provider can have it) — this
105
- matters most for OpenRouter, Ollama, LM Studio, vLLM, and any
106
- `openai-compat` endpoint, where the model actually loaded is opaque to the
107
- profile table. A caller that knows its model better should pass an explicit
108
- `toolMode` (agent layer) rather than rely on the capability default.
109
-
110
- **`listModels()` is optional per adapter — two engines never return anything.**
111
- `ProviderAdapter.listModels` is `?`-optional, and `AIHandler.listModels()`
112
- falls back to `[]` when an adapter doesn't implement it. Concretely: the
113
- `openaiChat` and `ollama` engines implement it (`GET /models` /
114
- `GET /api/tags`); the `anthropic` and `google` engines currently don't, so
115
- `handler.listModels()` for those providers always resolves to `[]`, not an
116
- error. This is easy to miss because `capabilities` doesn't surface it — an
117
- app building a model picker on top of `listModels()` should treat an empty
118
- result as "this provider doesn't support discovery," not "this provider has
119
- no models," and have a fallback (e.g. an app-configured default model per
120
- connection) ready for `anthropic`/`google` rather than showing an empty list.
121
-
122
- ### Letting an app's users pick a model (best practice)
123
-
124
- The nugget deliberately has no concept of "the app's currently selected
125
- model" — that's app policy, same as governance (see Non-goals). The pattern
126
- that has worked well for apps building a model-picker UI on top of the
127
- AI Nugget handler:
128
-
129
- 1. **Keep `provider`/`baseUrl` on a server-side allowlist; never take them
130
- from client input.** A `Connection`'s `provider`/`baseUrl` decide where
131
- the server's resolved API key gets sent (via `KeySource` → `applyAuth`);
132
- letting a client control either directly is effectively letting it
133
- redirect a real credential to an arbitrary endpoint (SSRF plus key
134
- exposure). Define your app's available connections server-side (e.g. a
135
- small config list, each with its own `KeyRef`), and only let the client
136
- choose a `connectionId` out of that list.
137
- 2. **`model` is fine as ordinary client input.** Unlike `provider`/`baseUrl`,
138
- a bad or unexpected `model` string just comes back as a normal provider
139
- error (unknown model) from `chat()`/`stream()` — there's no equivalent
140
- security concern, so there's no need to allowlist it server-side beyond
141
- what the provider itself enforces.
142
- 3. **Use `handler.listModels(connection)` to populate the picker live**, and
143
- fall back to an app-configured default model for connections whose engine
144
- doesn't implement discovery (see above).
145
-
146
- See `examples/model-picker.mjs` for a minimal runnable reference implementing
147
- this pattern.
148
-
149
- ## Recipes
150
-
151
- Small server apps built directly on `AIHandler` (see
152
- `examples/npm-mini-apps/`) tend to hit the same three points of friction:
153
- resolving connection config, turning a JSON-mode reply into a validated app
154
- object, and mapping a failure into an HTTP response. None of this is policy —
155
- it's just the boilerplate every consumer re-derives — so here's the
156
- canonical shape for each.
157
-
158
- ### Env-based connection setup
159
-
160
- `envConnection()` resolves a `Connection` plus a default `model` from
161
- app-owned env vars (`AI_PROVIDER`, `AI_MODEL`, `AI_KEY_ENV`, `AI_BASE_URL` by
162
- default):
53
+ CommonJS works too: `const { AIHandler } = require('@jxburros/ai-nugget')`.
163
54
 
164
- ```ts
165
- import { AIHandler, envConnection, envKeySource } from '@jxburros/ai-nugget';
166
-
167
- const handler = new AIHandler({ keySource: envKeySource() });
168
- const { connection, model } = envConnection({ id: 'my-app', defaultProvider: 'openai', defaultModel: 'gpt-4o-mini' });
169
-
170
- const result = await handler.chat(connection, { model, messages: [{ role: 'user', content: 'Hello!' }] });
171
- ```
55
+ For a running server, copy a starter from
56
+ [`examples/integrations/`](./examples/integrations/) — Express, Next.js,
57
+ Cloudflare Workers, or local Ollama.
172
58
 
173
- It only reads the server's own environment — `provider`/`baseUrl` never come
174
- from client input this way either, same as the manual pattern it replaces.
175
- Pass `env`, `providerVar`/`modelVar`/`keyEnvVar`/`baseUrlVar`, or
176
- `defaultProvider`/`defaultModel`/`defaultKeyEnv` to override the var names or
177
- fallbacks an app already uses.
59
+ ## Two things to configure before production
178
60
 
179
- ### JSON output + validation
61
+ Both are seams the library leaves open on purpose, and both are easy to miss:
180
62
 
181
- Ask for JSON in the prompt (and set `responseFormat: { type: 'json' }` for
182
- providers with `capabilities.jsonMode`), then run the raw text through
183
- `extractJsonWithSchema` with a small parse function built from the
184
- `require*` guards in `json.ts` — don't hand-roll a `/\{[\s\S]*\}/` regex plus
185
- `JSON.parse`, which silently accepts the first brace-looking substring and
186
- gives no useful error on a malformed or schema-incomplete reply:
63
+ 1. **Never take `provider` or `baseUrl` from client input.** They decide where
64
+ your resolved API key gets sent. Keep connections on a server-side allowlist
65
+ and let clients pick an id. → [security.md](./docs/security.md#ssrf-caller-controlled-baseurl)
66
+ 2. **Configure a `GovernancePolicy`.** With none, every provider and model is
67
+ allowed; the handler logs a startup notice so this is visible at runtime.
68
+ → [security.md](./docs/security.md#governance-is-neutral-by-default)
187
69
 
188
70
  ```ts
189
- import { extractJsonWithSchema, requireNumber, requireString } from '@jxburros/ai-nugget';
190
-
191
- function parseSprint(raw: unknown) {
192
- if (!raw || typeof raw !== 'object' || Array.isArray(raw)) throw new Error('Expected a JSON object');
193
- const record = raw as Record<string, unknown>;
194
- return { headline: requireString(record, 'headline'), minutes: requireNumber(record, 'minutes') };
195
- }
196
-
197
- const sprint = extractJsonWithSchema(result.text, parseSprint);
198
- ```
199
-
200
- `extractJsonWithSchema` recovers JSON from fenced code blocks or prose the
201
- model wrapped around it, and throws a typed `AIError` with
202
- `kind: 'invalid_response'` on anything that doesn't parse or doesn't match
203
- the schema — no JSON found, an array where an object was expected, a missing
204
- field, a string where a number belongs. That error flows into the same
205
- `AIError`-kind switch as any other provider failure (see below), so JSON
206
- validation doesn't need its own error path. `require*` covers scalars and
207
- string arrays only — validate array length, numeric ranges, or nested shapes
208
- yourself, same as `parseSprint` above; this is deliberately not a full
209
- schema validator (see Non-goals).
210
-
211
- ### Error handling matrix
212
-
213
- Every failure that reaches your `catch` around `handler.chat()`/`stream()` —
214
- including a JSON-validation failure from `extractJsonWithSchema` — is (or is
215
- normalized to) an `AIError` with a `kind`. Map `kind` to an HTTP status and a
216
- user-facing message once, log the raw `error.message` server-side, and never
217
- forward it to the client:
218
-
219
- | `kind` | Suggested status | Example user-facing message |
220
- |---|---|---|
221
- | `invalid_request` | 400 | "That request was invalid." |
222
- | `context_length` | 413 | "Your input is too long. Please shorten it and try again." |
223
- | `auth` | 502 | "The AI service is not configured correctly." |
224
- | `key_unavailable` | 500 | "The AI service is not configured correctly." |
225
- | `policy_blocked` | 403 | "This request was blocked by policy." |
226
- | `rate_limit` | 429 | "The AI service is busy. Please try again shortly." |
227
- | `timeout` | 504 | "The request took too long. Please try again." |
228
- | `canceled` | 499 | "The request was canceled." |
229
- | `invalid_response` | 502 | "The AI response wasn't in the expected format. Please try again." |
230
- | `network` / `server` | 502 | "The AI service is temporarily unavailable." |
231
- | `tool_error` | 500 | "Something went wrong running a tool." |
232
- | `budget_exceeded` | 429 | "This request used too many steps/tokens. Please try again." |
233
-
234
- `auth` and `key_unavailable` map to 5xx, not 401/403, because it's the
235
- *server's* credential that's wrong, not the caller's — the distinction
236
- matters for who should act on the error. `examples/npm-mini-apps/*/ai-error-map.mjs`
237
- is a copyable implementation of this table.
238
-
239
- ## What's implemented
240
-
241
- - **Core:** message-based contracts (`types.ts`), typed `AIError` + `classify()`,
242
- timeout/abort merging and SSE/NDJSON readers (`transport.ts`), defensive JSON
243
- extraction with array support + schema guards (`json.ts`), token estimation with
244
- an explicit `estimated` flag (`tokens.ts`).
245
- - **Adapters:** all four engines with streaming, native tool-calling, JSON modes,
246
- `finishReason` mapping, buffered-`stream:true` fallback, first-token latency,
247
- and usage normalization; the full v1 provider profile table. The
248
- OpenAI-compatible engine only sends `stream_options: { include_usage: true }`
249
- when the profile's `supportsUsageInStream` quirk confirms the server expects
250
- it — local servers (llama.cpp, LM Studio, vLLM) that don't have the quirk set
251
- never receive the option. The OpenAI and Azure OpenAI profiles send
252
- `max_completion_tokens` for `maxTokens`; local/OpenAI-compatible profiles keep
253
- `max_tokens` for compatibility. Google receives `responseSchema` for JSON mode
254
- only when native tools are absent, avoiding Gemini's JSON-with-tools rejection.
255
- - **Pipeline (`AIHandler`):** policy → keys → `beforeCall` → concurrency/min-interval
256
- → jittered retry (honoring `Retry-After`) → redacted `CallRecord` telemetry →
257
- `afterCall`, with `chat`, `stream`, `listModels`, `testConnection`. Every call —
258
- success, failure, or consumer-abandoned stream — produces one redacted telemetry
259
- record. Retries stop once user-visible output has been emitted, and telemetry
260
- or `afterCall` failures never re-execute an already-successful provider call.
261
- `listModels()` and `testConnection()` use explicit operation IDs
262
- (`__listModels__`, `__testConnection__`) so policy and telemetry can see those
263
- key-bearing probes too.
264
- - **Seams:** `KeySource` (`env`/`literal`/`memory`/chain + ref parsing),
265
- `Redactor` (default secret patterns + session-resolved key redaction),
266
- neutral `GovernancePolicy` (`blocklistPolicy`/`allowlistPolicy`/`composePolicies`).
267
- `allowlistPolicy` fails closed for providers omitted from the configured map;
268
- use `'*'` in a provider's prefix list to allow non-chat operation IDs.
269
- The default redactor covers common provider token formats and generic bearer
270
- tokens, with session-resolved keys always added exactly. `envConnection()`
271
- (`connect.ts`) resolves a `Connection` + default `model` from app-owned env
272
- vars — config-resolution convenience only, not a new seam or a policy
273
- default (see Recipes above).
274
- - **Agent layer (`@jxburros/ai-nugget/agent`):** `defineTool` + light JSON-schema
275
- arg validation (object-ness, `required`, top-level `properties[key].type` —
276
- not full JSON Schema: no `enum`, nested schemas, `oneOf`, bounds, `pattern`,
277
- array `items`, or `additionalProperties`; validate again inside
278
- `tool.execute` if you need stricter guarantees), `runAgent()` model↔tool loop
279
- over the full handler pipeline,
280
- streamed `AgentEvent`s, budgets (`maxSteps`/`maxTokens`/`deadlineMs`) with honest
281
- `stopReason`s, `ApprovalGate` for side-effecting tools (deny is fed back to the
282
- model as data), and `native` / `promptJson` / `auto` tool modes. `auto` (the
283
- default) resolves per call from the connection's provider capability profile
284
- (`profileFor(provider).capabilities.nativeTools`) — hosted providers get
285
- `native`, local runtimes and the `openai-compat` escape hatch get
286
- `promptJson`, and an explicit `toolMode` always overrides the default.
287
- `promptJson` mode accepts a single `{"tool","input"}` directive or a batched
288
- `{"tools":[…]}` (a bare array works too), so the model can request several
289
- tools in one turn just like native tool-calling. Prompt-JSON history is
290
- serialized as plain text turns, not provider-native tool-call wire format.
291
-
292
- ```ts
293
- import { runAgent, defineTool } from '@jxburros/ai-nugget/agent';
71
+ new AIHandler({
72
+ keySource: envKeySource(),
73
+ policy: allowlistPolicy({ openai: ['gpt-4o', 'gpt-4o-mini'], ollama: ['*'] }),
74
+ });
294
75
  ```
295
76
 
296
- ## Commands
77
+ ## Documentation
78
+
79
+ | Doc | Covers |
80
+ |---|---|
81
+ | [providers.md](./docs/providers.md) | The 19 profiles, which are **live-verified vs mock-verified vs config-only**, capabilities, JSON-mode/tools limits, `openai-compat`, model discovery |
82
+ | [reliability.md](./docs/reliability.md) | `timeoutMs` vs `idleTimeoutMs`, why retries stop after output is emitted, double-billing and idempotency, why limits are per-instance |
83
+ | [security.md](./docs/security.md) | Key handling, redaction coverage, SSRF, opt-in injection defenses, the secure-tool recipe |
84
+ | [agent-loop.md](./docs/agent-loop.md) | `runAgent`, tool modes, budgets and what happens mid-step, the approval gate |
85
+ | [recipes.md](./docs/recipes.md) | Env-based connections, JSON output + validation, the error-handling matrix, telemetry |
86
+ | [integrations.md](./docs/integrations.md) | The host matrix CI runs (Node, Next.js, Workers, local runtime, packaging) |
87
+ | [distribution.md](./docs/distribution.md) | Commands, live smoke tests, CI workflows, npm/GitHub Packages/vendored `nugget/`, bundler notes |
88
+ | [design.md](./design.md) | The full contract |
89
+ | [state-of-the-nugget.md](./docs/state-of-the-nugget.md) | Current health snapshot: architecture, safety and reliability posture, coverage gaps, what's next |
90
+ | [UPGRADING.md](./UPGRADING.md) | Version-by-version changes |
91
+
92
+ ## What's in the box
93
+
94
+ - **Core** — message-based contracts (`types.ts`), typed `AIError` + `classify()`
95
+ with wire-boundary redaction, timeout/idle-timeout/abort merging and SSE/NDJSON
96
+ readers (`transport.ts`), defensive JSON extraction with schema guards
97
+ (`json.ts`), token estimation with an explicit `estimated` flag.
98
+ - **Adapters** — four engines (`openaiChat`, `anthropic`, `google`, `ollama`)
99
+ with streaming, native tool-calling, JSON modes, `finishReason` mapping,
100
+ buffered-`stream:true` fallback, reasoning-token channel, first-token latency,
101
+ usage normalization, and truncation detection (`stream_anomaly`). Provider
102
+ differences live in a data-driven profile table, not in copies of the SSE loop.
103
+ - **Pipeline (`AIHandler`)** — `chat`, `stream`, `chatParsed`, `embed`,
104
+ `listModels`, `testConnection`, `prewarm`. Every call — success, failure, or
105
+ abandoned stream — produces exactly one redacted `CallRecord`. Retries are
106
+ jittered and honor `Retry-After`; telemetry or `afterCall` failures never
107
+ re-run a completed provider call.
108
+ - **Seams** — `KeySource` (env/literal/memory/chain + ref parsing), `Redactor`,
109
+ neutral `GovernancePolicy` (`blocklistPolicy`/`allowlistPolicy`/`composePolicies`),
110
+ `TelemetrySink`, and an optional `pricing` hook. `providerOptions` passes
111
+ provider-native fields through without waiting for a library release;
112
+ `reasoningEffort` is one word for "how hard should the model think", mapped
113
+ per provider ([docs](./docs/providers.md#reasoning-effort)).
114
+ - **Agent layer** (`@jxburros/ai-nugget/agent`) — `defineTool`, `runAgent()`
115
+ model↔tool loop over the full pipeline, streamed `AgentEvent`s, budgets with
116
+ honest `stopReason`s, an `ApprovalGate` for side-effecting tools, and
117
+ `native`/`promptJson`/`auto` tool modes.
118
+
119
+ ## What's changed since 0.6.0
120
+
121
+ **0.7.0 is backward-compatible for ordinary consumers** — the first two changes are additive; the third changes what `delta` carries for reasoning models (see UPGRADING). Full detail in the
122
+ [changelog](./CHANGELOG.md) and [UPGRADING.md](./UPGRADING.md#06x--070).
123
+
124
+ - **`ChatRequest.reasoningEffort`** (`none` / `minimal` / `low` / `medium` /
125
+ `high`) — one vocabulary for reasoning effort, mapped onto OpenAI
126
+ `reasoning_effort`, Anthropic `thinking`, Google `thinkingConfig`, and Ollama
127
+ `think`; forwarded by the agent loop via `AgentOptions.reasoningEffort`.
128
+ Never sent unless you set it. See [Reasoning effort](./docs/providers.md#reasoning-effort).
129
+ - **OpenAI reasoning models can call tools again.** `/chat/completions` refuses
130
+ function tools unless `reasoning_effort` is `'none'`; the `openaiChat` engine
131
+ now retries that one request with it set and emits a
132
+ `reasoning_effort_disabled_for_tools` context event so the change is visible.
133
+ Learned from AI Server Studio's Build 3 shakedown, where every tool-using
134
+ turn on `gpt-5.x`/`gpt-6` 400'd.
135
+ - **Inline `<think>` reasoning never reaches `delta` or `result.text`.** The
136
+ handler strips it on every engine and emits it as `reasoning` events
137
+ (`stripInlineReasoning: false` opts out). Ported from the stripper AI Server
138
+ Studio had to run at every call site. See [Inline reasoning](./docs/providers.md#inline-reasoning-is-stripped-at-the-seam).
139
+
140
+ ## What's changed since 0.5.0
141
+
142
+ **0.6.0 is backward-compatible** — every change below is additive. The one thing
143
+ to check is the new `not_found` error kind if you `switch` exhaustively over
144
+ `AIErrorKind`; see [UPGRADING.md](./UPGRADING.md#05x--060). Full detail lives in
145
+ the [changelog](./CHANGELOG.md).
146
+
147
+ - **Security & correctness** — `beforeCall` hooks now receive a scrubbed
148
+ connection (masked literal `keyRef` and auth headers, safe to log);
149
+ `classify()` redacts provider response excerpts at the wire boundary so an
150
+ `AIError` can never carry an unredacted secret; label-anchored redaction
151
+ covers unprefixed secrets (Azure `api-key`, AWS keys, `client_secret`, session
152
+ tokens); a new `not_found` error kind distinguishes 404/410 from a malformed
153
+ request; and Ollama tool-call arguments coerce through the shared parser so a
154
+ llama.cpp-style backend sending JSON-as-string no longer fails.
155
+ - **Observability** — `stream_anomaly` detection now fires on all four engines
156
+ (not just `openaiChat`); Google and Anthropic emit `json_mode_downgraded` when
157
+ a JSON mode is dropped for tools; `AIHandler` logs a startup notice when no
158
+ `GovernancePolicy` is configured; and OpenAI requests carry a stable
159
+ `Idempotency-Key` across retries to mitigate double-billing.
160
+ - **Providers & types** — `openai-compat` is now `keyOptional`, matching the
161
+ other local-runtime profiles, so keyless servers (including JX Runtime) work
162
+ without a `keyRef`; `Connection.provider` is typed `KnownProvider | (string &
163
+ {})` for autocomplete and typo-catching with the escape hatch still open.
164
+ - **Docs & examples** — `README.md` was trimmed to a front door with reference
165
+ material moved under [`docs/`](./docs/) (`providers`, `reliability`,
166
+ `security`, `agent-loop`, `recipes`, `integrations`, `distribution`), and
167
+ [`examples/integrations/`](./examples/integrations/) added five CI-verified
168
+ starters (Express, Next.js, Cloudflare Workers, local Ollama, ESM/CJS
169
+ packaging check).
170
+
171
+ ## Contributing
172
+
173
+ `AGENTS.md` is the always-on baseline — invariants, required reading, and the
174
+ [admission test](./AGENTS.md#feature-admission-test) a new feature has to pass.
175
+ `.claude/skills/` holds task-specific guides (`use-ai-nugget`,
176
+ `build-agent-loop`, `add-provider`, `develop-nugget`) that Claude Code loads
177
+ automatically and any agent or human can read as plain Markdown.
297
178
 
298
179
  ```bash
299
- npm install
300
- npm test # Vitest contract suite in Node (97 tests; live tests skipped unless env-gated)
301
- npx playwright install chromium # one-time, before test:browser
302
- npm run test:browser # same suite in headless Chromium (proves isomorphism)
303
- npm run build # tsc → dist/ (also the typecheck)
304
- npm run build:nugget # writes a vendorable nugget/ stamped with version + content hash
180
+ npm test # Node contract suite
181
+ npm run test:browser # same suite in headless Chromium (isomorphism)
182
+ npm run build && npm run build:nugget
305
183
  ```
306
184
 
307
- Requires Node `^20.19.0` or `>=22.12.0` (see `engines` in `package.json` and
308
- `.nvmrc`) — the toolchain's build step (`vitest` → `rolldown`) needs a native
309
- binding unavailable on older Node 20 patch releases.
310
-
311
- ### Live smoke tests (optional, env-gated)
312
-
313
- `tests/live-smoke.test.ts` exercises the real wire path against a live provider.
314
- It is **skipped unless `AI_HANDLER_LIVE=1`**, so it never runs in the default
315
- suite or in CI:
316
-
317
- ```bash
318
- # Local Ollama (defaults: provider=ollama, model=llama3.2)
319
- AI_HANDLER_LIVE=1 npm run test:live
320
-
321
- # A cloud provider, with JSON-mode and agent tool-loop checks enabled
322
- AI_HANDLER_LIVE=1 AI_HANDLER_LIVE_PROVIDER=openai AI_HANDLER_LIVE_MODEL=gpt-4o-mini \
323
- AI_HANDLER_LIVE_KEY_ENV=OPENAI_API_KEY AI_HANDLER_LIVE_JSON=1 AI_HANDLER_LIVE_TOOLS=1 \
324
- npm run test:live
325
- ```
326
-
327
- Config env vars: `AI_HANDLER_LIVE_PROVIDER`, `AI_HANDLER_LIVE_MODEL`,
328
- `AI_HANDLER_LIVE_BASE_URL`, `AI_HANDLER_LIVE_KEY` (literal) or
329
- `AI_HANDLER_LIVE_KEY_ENV` (key from an env var), `AI_HANDLER_LIVE_JSON`,
330
- `AI_HANDLER_LIVE_TOOLS`.
331
-
332
- **Live provider matrix (`.github/workflows/live-matrix.yml`).** The default
333
- CI workflow only exercises mocked fetch responses, so subtle live-wire
334
- behavior differences (OpenAI, Anthropic, Google, OpenRouter, and local
335
- runtimes each drift independently) can't surface there. `live-matrix.yml`
336
- runs the same `tests/live-smoke.test.ts` suite against real endpoints —
337
- manually (`workflow_dispatch`) or on a weekly schedule — never on push/PR, so
338
- it can't block or flake a normal CI run. Each matrix entry reads its API key
339
- from a same-named repository secret (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`,
340
- `GOOGLE_API_KEY`, `OPENROUTER_API_KEY`) and skips itself with a notice if that
341
- secret isn't configured, so you only need to add secrets for the providers
342
- you actually use. Local runtimes (Ollama, llama.cpp, LM Studio, vLLM) aren't
343
- in the matrix since they need a reachable server; smoke-test those manually
344
- with the local `test:live` invocation above.
345
-
346
- ## Distribution
347
-
348
- Three supported paths, in order of preference:
349
-
350
- 1. **npm registry (primary).** `@jxburros/ai-nugget` publishes to
351
- [npmjs.org](https://www.npmjs.com/package/@jxburros/ai-nugget) on every
352
- published GitHub release (`.github/workflows/publish.yml`). Consumers install
353
- from the normal npm registry:
354
-
355
- ```bash
356
- npm install @jxburros/ai-nugget@^0.3.1
357
- ```
358
-
359
- This is the default dependency path for apps: fix once here, bump the
360
- dependency in each app, and let npm resolve it normally.
361
- 2. **GitHub Packages (kept).** The same release workflow also publishes
362
- `@jxburros/ai-nugget` to GitHub Packages for GitHub-native workflows and
363
- existing portfolio consumers that already point at that registry. Those apps
364
- add a project `.npmrc`:
365
-
366
- ```
367
- @jxburros:registry=https://npm.pkg.github.com
368
- //npm.pkg.github.com/:_authToken=${GITHUB_TOKEN}
369
- ```
370
-
371
- (GitHub Packages requires an authenticated token even for public-repo
372
- packages.) Then `npm install @jxburros/ai-nugget@^0.3.1`.
373
- 3. **Vendored `nugget/` (fallback).** `nugget/` is a generated single-folder
374
- build (`src/` + `VERSION.txt` with a version + content-hash stamp) for repos
375
- that cannot take a package dependency. Copy it in; `VERSION.txt` makes drift
376
- from the source of truth detectable. (Bundlers that don't resolve
377
- `.ts`+`.js`-suffixed imports the way `tsc` does — e.g. Turbopack — should
378
- vendor `dist/` instead; see below.)
379
-
380
- `dist/` (ESM + `.d.ts`) is the package's own build output. `dist/` and
381
- `nugget/` are both committed and regenerated from `src/`; `prepublishOnly`
382
- rebuilds both, and CI fails if either is stale.
383
-
384
- **Bundler compatibility for the vendored `nugget/` path.** `nugget/src/*.ts`
385
- uses NodeNext-style relative imports with explicit `.js` extensions (e.g.
386
- `export * from './types.js'`), which `tsc` needs to resolve `.ts` files under
387
- `moduleResolution: bundler`/`nodenext`. Not every bundler's runtime module
388
- graph treats `.ts`/`.js` as interchangeable the way `tsc` does — confirmed
389
- with Next.js 16's Turbopack: aliasing straight at `nugget/src/index.ts`
390
- type-checks but fails to resolve at build/dev time (`Module not found: Can't
391
- resolve './types.js'`), and aliasing at the compiled `dist/index.d.ts` builds
392
- but silently resolves some named exports to `undefined` at runtime. If your
393
- bundler hits this, vendor `dist/` (real `.js` + `.d.ts` pairs) instead of
394
- `nugget/`, and point path aliases at its `.js` entry points (e.g.
395
- `dist/index.js`, `dist/agent/index.js`) — TypeScript picks up the sibling
396
- `.d.ts` automatically.
185
+ Full command list and CI layout: [distribution.md](./docs/distribution.md).
397
186
 
398
187
  ## Examples
399
188
 
400
- `examples/` has runnable scripts against local Ollama and llama.cpp servers,
401
- `promptJson` vs. native tool-calling, an `ApprovalGate`, telemetry, and a small
402
- industrial-themed chatbot demo under `examples/steel-chat/` — see
189
+ `examples/` has runnable scripts (local Ollama, llama.cpp, `promptJson` vs
190
+ native tools, approval gates, telemetry, a model picker), a small chat demo
191
+ (`npm run demo:steel-chat`), three mini-apps consuming the published package,
192
+ and the CI-verified [integration starters](./examples/integrations/). See
403
193
  `examples/README.md`.
404
194
 
405
- If you want a compact end-to-end app example instead of a single script, run:
406
-
407
- ```bash
408
- npm run demo:steel-chat
409
- ```
195
+ ## Non-goals
410
196
 
411
- That demo exposes a simple browser chat UI backed by `AIHandler`, discovers
412
- allowed connections/models server-side, and displays the current package
413
- version in the interface so the example stays aligned with the installed nugget.
197
+ Not a full agent framework (no planning/memory/RAG/multi-agent), not a
198
+ router/recommender (apps choose models), not a secrets vault (`KeySource` is a
199
+ seam), and not a UI. The governance seam ships neutral: it is where an app *can*
200
+ enforce rules, not a place the library imposes its own.
414
201
 
415
- ## Agent skills
202
+ <!-- GitHub Pages deployment is configured in .github/workflows/pages.yml. -->
416
203
 
417
- `.claude/skills/` contains Agent Skills — task-specific guides that Claude Code
418
- picks up automatically and that any AI agent (or human) can read as plain
419
- Markdown: `use-ai-nugget` (integrating the nugget into an app),
420
- `build-agent-loop` (the agent/tool-calling layer), `add-provider` (extending
421
- the provider profile table), and `develop-nugget` (invariants and validation
422
- for changes to this repo). `AGENTS.md` remains the always-on baseline; the
423
- skills carry the per-task detail.
204
+ ---
424
205
 
425
- ## Non-goals
206
+ ## Project Stewardship
426
207
 
427
- Not a full agent framework (no planning/memory/RAG/multi-agent — the loop
428
- consumes `messages` and stops there), not a router/recommender (apps choose
429
- models; the handler exposes a `route`/`context` event so apps can report what they
430
- chose), not a secrets vault (`KeySource` is a seam), and not a UI. The governance
431
- seam ships neutral: it is where an app *can* enforce rules, not a place the
432
- library imposes its own.
208
+ This project is developed by **Jeffrey** through **JX Holdings, LLC**. GitHub: [`jxburros`](https://github.com/jxburros).