@navels/neal 0.2.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +206 -209
- package/SECURITY.md +16 -19
- package/dist/neal/adjudicator/{blocked-adjudicator.js → consultant.js} +21 -21
- package/dist/neal/adjudicator/execute.js +33 -33
- package/dist/neal/adjudicator/final-completion.js +2 -31
- package/dist/neal/adjudicator/planning.js +230 -38
- package/dist/neal/agents/prompts.js +6 -17
- package/dist/neal/agents/rounds.js +43 -73
- package/dist/neal/agents/schemas.js +47 -13
- package/dist/neal/blocked-guidance.js +16 -9
- package/dist/neal/cli.js +2 -2
- package/dist/neal/commands/check.js +2 -2
- package/dist/neal/commands/compat.js +100 -36
- package/dist/neal/commands/interactive-activity.js +2 -2
- package/dist/neal/commands/recovery-guidance.js +20 -8
- package/dist/neal/commands/runtime.js +4 -2
- package/dist/neal/config.js +16 -7
- package/dist/neal/context/inline-review-context.js +13 -28
- package/dist/neal/context/reviewer-context.js +19 -14
- package/dist/neal/eval/reviewer-eval-manifest.js +86 -0
- package/dist/neal/eval/reviewer-eval.js +155 -0
- package/dist/neal/interactive-controls.js +5 -0
- package/dist/neal/orchestrator/completion.js +49 -2
- package/dist/neal/orchestrator/failures.js +7 -0
- package/dist/neal/orchestrator/notifications.js +9 -9
- package/dist/neal/orchestrator/phases/planning.js +117 -80
- package/dist/neal/orchestrator/phases/recovery.js +151 -115
- package/dist/neal/orchestrator/phases/review.js +28 -1
- package/dist/neal/orchestrator/transitions.js +7 -1
- package/dist/neal/orchestrator.js +3 -0
- package/dist/neal/plan-queue.js +29 -1
- package/dist/neal/prompts/execute.js +7 -19
- package/dist/neal/prompts/planning.js +64 -55
- package/dist/neal/prompts/review-doctrine.js +13 -50
- package/dist/neal/prompts/specialized.js +7 -17
- package/dist/neal/prompts/specs.js +180 -32
- package/dist/neal/providers/agent-settings-isolation.js +35 -0
- package/dist/neal/providers/anthropic-claude.js +18 -1
- package/dist/neal/providers/detection.js +3 -7
- package/dist/neal/providers/git-config-isolation.js +83 -0
- package/dist/neal/providers/openai-codex.js +25 -3
- package/dist/neal/providers/{generic-agentic-tools.js → openai-compatible-tools.js} +11 -5
- package/dist/neal/providers/openai-compatible.js +1078 -430
- package/dist/neal/providers/openrouter-routing.js +68 -0
- package/dist/neal/providers/pricing.js +1 -1
- package/dist/neal/providers/rate-card.js +6 -32
- package/dist/neal/providers/registry.js +6 -17
- package/dist/neal/providers/types.js +12 -0
- package/dist/neal/resume-decision.js +7 -6
- package/dist/neal/resume-planner.js +9 -0
- package/dist/neal/retrospective.js +3 -3
- package/dist/neal/review-debt.js +30 -0
- package/dist/neal/review-findings/provider.js +17 -29
- package/dist/neal/review.js +17 -1
- package/dist/neal/state-invariants.js +11 -3
- package/dist/neal/state-views.js +27 -5
- package/dist/neal/state.js +59 -15
- package/dist/neal/status.js +28 -1
- package/dist/neal/support.js +3 -3
- package/docs/README.md +22 -18
- package/docs/{ADJUDICATOR_INVENTORY.md → adjudicator-inventory.md} +58 -57
- package/docs/architecture.md +79 -83
- package/docs/automation.md +11 -11
- package/docs/compat.md +91 -61
- package/docs/compatible-models.md +145 -121
- package/docs/demo.md +15 -5
- package/docs/maintenance.md +65 -34
- package/docs/plan-format.md +61 -33
- package/docs/prompt-evals.md +96 -0
- package/docs/{PROMPT_SPECS.md → prompt-specs.md} +52 -48
- package/docs/providers.md +279 -349
- package/docs/release.md +63 -81
- package/docs/review-convergence.md +246 -0
- package/docs/state-machine.md +61 -59
- package/docs/storage.md +40 -31
- package/docs/troubleshooting.md +32 -29
- package/examples/compat/add-edit-verify/PLAN.md +2 -0
- package/examples/compat/add-edit-verify/broken.diff +2 -2
- package/examples/compat/add-edit-verify/good.diff +2 -2
- package/examples/compat/add-edit-verify/src/add.js +1 -1
- package/examples/compat/manifest.json +25 -29
- package/examples/compat/sum-grep-edit/PLAN.md +33 -0
- package/examples/compat/sum-grep-edit/broken.diff +12 -0
- package/examples/compat/sum-grep-edit/good.diff +12 -0
- package/examples/compat/sum-grep-edit/package.json +5 -0
- package/examples/compat/sum-grep-edit/src/numbers.js +9 -0
- package/examples/compat/sum-grep-edit/test/numbers.test.js +15 -0
- package/examples/issue-triage-js/README.md +0 -15
- package/examples/reviewer-eval/clean-doc/base/src/clamp.ts +3 -0
- package/examples/reviewer-eval/clean-doc/change.diff +9 -0
- package/examples/reviewer-eval/clean-extract/base/src/greet.ts +3 -0
- package/examples/reviewer-eval/clean-extract/change.diff +13 -0
- package/examples/reviewer-eval/dropped-error/base/src/load.ts +4 -0
- package/examples/reviewer-eval/dropped-error/change.diff +15 -0
- package/examples/reviewer-eval/manifest.json +45 -0
- package/examples/reviewer-eval/off-by-one/base/src/paginate.ts +5 -0
- package/examples/reviewer-eval/off-by-one/change.diff +11 -0
- package/{config.yml → neal.yml} +1 -1
- package/package.json +15 -14
- package/dist/neal/providers/generic-agentic.js +0 -1256
- package/docs/comparison.md +0 -105
- package/docs/issue-pipeline.md +0 -124
- package/examples/compat/is-even-add-test/PLAN.md +0 -30
- package/examples/compat/is-even-add-test/broken.diff +0 -11
- package/examples/compat/is-even-add-test/good.diff +0 -11
- package/examples/compat/is-even-add-test/package.json +0 -5
- package/examples/compat/is-even-add-test/src/is-even.js +0 -3
- package/examples/compat/is-even-add-test/test/is-even.test.js +0 -9
- package/examples/compat/reverse-grep-edit/PLAN.md +0 -32
- package/examples/compat/reverse-grep-edit/broken.diff +0 -12
- package/examples/compat/reverse-grep-edit/good.diff +0 -12
- package/examples/compat/reverse-grep-edit/package.json +0 -5
- package/examples/compat/reverse-grep-edit/src/strings.js +0 -10
- package/examples/compat/reverse-grep-edit/test/strings.test.js +0 -15
package/docs/providers.md
CHANGED
|
@@ -1,25 +1,22 @@
|
|
|
1
|
-
# Provider
|
|
1
|
+
# Provider adapters
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
neal's writer-run provider layer is an internal extension point for built-in
|
|
4
4
|
agent SDK adapters. A provider adapter maps one SDK's sessions, tools,
|
|
5
5
|
structured-output behavior, errors, usage data, and stream/message events into
|
|
6
|
-
|
|
6
|
+
neal-owned contracts.
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
neal currently ships built-in adapters for:
|
|
9
9
|
|
|
10
10
|
- `openai-codex`
|
|
11
11
|
- `anthropic-claude`
|
|
12
|
-
- `openai-compatible
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
[Generic Agentic Writer](#generic-agentic-writer))
|
|
19
|
-
|
|
20
|
-
Neal does not currently load external provider plugins or discover
|
|
12
|
+
- `openai-compatible`, a neal-owned agent loop for any OpenAI-compatible
|
|
13
|
+
tool-calling endpoint, with writer tools for the coder and read-only tools
|
|
14
|
+
for the reviewer. See
|
|
15
|
+
[OpenAI-compatible provider](#openai-compatible-provider).
|
|
16
|
+
|
|
17
|
+
neal does not currently load external provider plugins or discover
|
|
21
18
|
npm-installed providers. Public `neal review` is a separate read-only findings
|
|
22
|
-
flow that uses configured provider adapters
|
|
19
|
+
flow that uses configured provider adapters. It is not a writer-run mode and is
|
|
23
20
|
not a registered provider id.
|
|
24
21
|
|
|
25
22
|
`neal setup` can inspect local built-in provider runtime availability while
|
|
@@ -29,7 +26,7 @@ entrypoint and Claude SDK-bundled or standalone executable paths, but it does
|
|
|
29
26
|
not authenticate, send prompts, collect secrets, or verify account state. Use
|
|
30
27
|
`neal check` after setup for the live provider connectivity prompt.
|
|
31
28
|
|
|
32
|
-
## Contract
|
|
29
|
+
## Contract overview
|
|
33
30
|
|
|
34
31
|
Provider contracts live in `src/neal/providers/types.ts`. A built-in provider
|
|
35
32
|
module should export a `NealProviderDefinition` plus any focused test hooks it
|
|
@@ -50,7 +47,7 @@ checks all resolve provider identity through that registry.
|
|
|
50
47
|
|
|
51
48
|
## Roles
|
|
52
49
|
|
|
53
|
-
|
|
50
|
+
neal exposes three effective agent roles for writer runs:
|
|
54
51
|
|
|
55
52
|
- `planner`: writes and revises plan artifacts through the coder adapter surface.
|
|
56
53
|
It inherits the coder provider/model unless configured explicitly.
|
|
@@ -59,8 +56,8 @@ Neal exposes three effective agent roles for writer runs:
|
|
|
59
56
|
- `reviewer`: runs schema-oriented rounds for review, plan review, support, and
|
|
60
57
|
final completion review work through the structured-advisor adapter surface.
|
|
61
58
|
|
|
62
|
-
Provider capability roles remain `coder` and `structured-advisor
|
|
63
|
-
a
|
|
59
|
+
Provider capability roles remain `coder` and `structured-advisor`. `planner` is
|
|
60
|
+
a neal role label backed by the configured provider's coder adapter. The same
|
|
64
61
|
provider may support both capability roles, or only one. The checked-in config
|
|
65
62
|
template and first-run setup examples use `openai-codex` for the coder role and
|
|
66
63
|
`anthropic-claude` for the reviewer role, but fresh writer runs still require
|
|
@@ -77,17 +74,16 @@ Each provider capability role declares:
|
|
|
77
74
|
- read, write, and shell tool access
|
|
78
75
|
- session resume support
|
|
79
76
|
- model override support
|
|
80
|
-
-
|
|
77
|
+
- neal structured control protocol support
|
|
81
78
|
- usage reporting support
|
|
82
79
|
|
|
83
|
-
Capabilities are enforced before writer work starts or resumes.
|
|
80
|
+
Capabilities are enforced before writer work starts or resumes. neal requires
|
|
84
81
|
the planner and coder effective roles to resolve to providers with the coder
|
|
85
|
-
capability, write and shell access, and
|
|
82
|
+
capability, write and shell access, and neal structured control protocol support.
|
|
86
83
|
The reviewer effective role must resolve to a provider with the
|
|
87
|
-
structured-advisor capability and
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
required when a persisted session handle is present.
|
|
84
|
+
structured-advisor capability and read tool access: every reviewer inspects
|
|
85
|
+
the repository directly. Session resume support is required when a persisted
|
|
86
|
+
session handle is present.
|
|
91
87
|
|
|
92
88
|
Current built-in capabilities are intentionally conservative:
|
|
93
89
|
|
|
@@ -95,35 +91,30 @@ Current built-in capabilities are intentionally conservative:
|
|
|
95
91
|
read, write, and shell capable because the current SDK configuration uses
|
|
96
92
|
broad local access. The structured-advisor (reviewer) role is read capable
|
|
97
93
|
but never write or shell capable (see
|
|
98
|
-
[The
|
|
94
|
+
[The read-only reviewer invariant](#the-read-only-reviewer-invariant)).
|
|
99
95
|
- Anthropic Claude supports coder and structured-advisor roles. The coder role
|
|
100
96
|
is read, write, and shell capable. The structured-advisor (reviewer) role is
|
|
101
97
|
read capable but never write or shell capable.
|
|
102
|
-
- OpenAI-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
The structured-advisor role is read capable (a read-only toolset; no write
|
|
109
|
-
or shell access), so a generic reviewer inspects the repository directly
|
|
110
|
-
with read tools instead of receiving Neal-inlined context. Neither role
|
|
111
|
-
supports session resume. See
|
|
112
|
-
[Generic Agentic Writer](#generic-agentic-writer).
|
|
98
|
+
- OpenAI-compatible supports coder and structured-advisor roles. The coder
|
|
99
|
+
role is read, write, and shell capable through neal-owned tools executed
|
|
100
|
+
locally. The structured-advisor role is read capable (a read-only toolset,
|
|
101
|
+
no write or shell access), so its reviewer inspects the repository directly
|
|
102
|
+
with read tools. Neither role supports session resume. See
|
|
103
|
+
[OpenAI-compatible provider](#openai-compatible-provider).
|
|
113
104
|
|
|
114
105
|
The `coder` capability describes adapter paths, not a global promise that every
|
|
115
106
|
writer-run workflow is read-only: writer-run coder turns use provider SDKs with
|
|
116
107
|
broad local permissions. The separate public `neal review` command enforces its
|
|
117
108
|
own read-only boundary in the review command flow.
|
|
118
109
|
|
|
119
|
-
### Coder
|
|
110
|
+
### Coder tool policy enforcement
|
|
120
111
|
|
|
121
112
|
Plan-authoring rounds pass a `toolPolicy` (`allowedWritePaths` restricted to
|
|
122
113
|
the plan document, `allowRun: false`) to the coder adapter. Enforcement
|
|
123
114
|
strength is adapter-specific and mechanical where each SDK allows it:
|
|
124
115
|
|
|
125
|
-
- `
|
|
126
|
-
toolset
|
|
116
|
+
- `openai-compatible`: full jail. The policy selects the neal-owned plan-author
|
|
117
|
+
toolset: writes only to the allowlisted paths, no shell tool at all.
|
|
127
118
|
- `anthropic-claude`: tool exclusion plus a path callback. `allowRun: false`
|
|
128
119
|
removes Bash from the turn's tools list, and a PreToolUse hook denies any
|
|
129
120
|
write-class tool call whose resolved path is not allowlisted.
|
|
@@ -136,27 +127,26 @@ strength is adapter-specific and mechanical where each SDK allows it:
|
|
|
136
127
|
Rounds without a `toolPolicy` are unaffected on every adapter: ordinary coder
|
|
137
128
|
scope rounds keep full access because verification legitimately runs commands.
|
|
138
129
|
|
|
139
|
-
### The
|
|
130
|
+
### The read-only reviewer invariant
|
|
140
131
|
|
|
141
|
-
A reviewer exists to *judge*, never to *modify* or *re-execute*.
|
|
132
|
+
A reviewer exists to *judge*, never to *modify* or *re-execute*. neal enforces
|
|
142
133
|
this structurally: every *supported* `structured-advisor` capability has
|
|
143
134
|
`write: false` and `shell: false`. The invariant constrains only `write` and
|
|
144
|
-
`shell
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
Neal-inlined context rather than read tools).
|
|
135
|
+
`shell`. `read` is not part of the invariant, but every built-in reviewer
|
|
136
|
+
(`openai-codex`, `anthropic-claude`, `openai-compatible`) declares read access,
|
|
137
|
+
and the writer-run capability check requires it.
|
|
148
138
|
|
|
149
139
|
`src/neal/providers/registry.ts` asserts this over every registered provider
|
|
150
|
-
definition
|
|
140
|
+
definition (built-in or test-registered) at the definition-resolution
|
|
151
141
|
chokepoint and at test registration time, so no provider definition can ever
|
|
152
142
|
declare a writing or shell-running reviewer. The assertion checks only the
|
|
153
|
-
`write`/`shell` half
|
|
143
|
+
`write`/`shell` half. It never constrains `read`, and it never touches the
|
|
154
144
|
`coder` capability (coders keep `write`/`shell`).
|
|
155
145
|
|
|
156
146
|
Two consequences follow from no reviewer holding shell access:
|
|
157
147
|
|
|
158
148
|
- Reviewers never re-run tests or other verification. Verification happens once,
|
|
159
|
-
performed by the coder
|
|
149
|
+
performed by the coder. The reviewer trusts the coder's reported result plus
|
|
160
150
|
its own reading of the diff and (for read-tool reviewers) the repository.
|
|
161
151
|
- Reviewers never mutate the checkout. A review produces a verdict and findings,
|
|
162
152
|
not edits.
|
|
@@ -164,23 +154,23 @@ Two consequences follow from no reviewer holding shell access:
|
|
|
164
154
|
Usage reporting is `opportunistic`: providers emit usage only when the SDK
|
|
165
155
|
event or result supplies it.
|
|
166
156
|
|
|
167
|
-
### Review
|
|
157
|
+
### Review doctrine access modes
|
|
168
158
|
|
|
169
|
-
Reviewer prompts render
|
|
159
|
+
Reviewer prompts render neal's shared review doctrine in one of two access
|
|
170
160
|
modes, derived from the configured reviewer provider's structured-advisor
|
|
171
161
|
tool access:
|
|
172
162
|
|
|
173
|
-
- `read-only` (read access without shell access
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
|
|
178
|
-
|
|
179
|
-
- `
|
|
163
|
+
- `read-only` (read access without shell access, which every built-in reviewer
|
|
164
|
+
today): the reviewer is instructed to inspect the repository through read
|
|
165
|
+
tools only (reading changed files and searching the tree) and is never
|
|
166
|
+
instructed to run commands, run tests, or use scratch directories.
|
|
167
|
+
Commit-range visibility depends on whether the provider exposes its own
|
|
168
|
+
range-diff tool:
|
|
169
|
+
- `openai-compatible` declares a read-only `git_diff` commit-range tool
|
|
180
170
|
(`providesRangeDiffTool`), so its reviewer inspects the range with that tool
|
|
181
171
|
and receives no inlined diff.
|
|
182
172
|
- native read-only reviewers such as `openai-codex` and `anthropic-claude`
|
|
183
|
-
have read and search tools but no commit-range diff tool, so
|
|
173
|
+
have read and search tools but no commit-range diff tool, so neal inlines
|
|
184
174
|
the commit-range diff into the reviewer prompt as the source of truth for
|
|
185
175
|
what the range changed (including deletions and renames that head-state file
|
|
186
176
|
reads cannot reveal), and the reviewer uses its read/search tools to verify
|
|
@@ -189,28 +179,23 @@ tool access:
|
|
|
189
179
|
In both branches the doctrine states that absence from a diff is not evidence
|
|
190
180
|
of absence from the repository: the reviewer must open files with read tools
|
|
191
181
|
before claiming a missing import or declaration.
|
|
192
|
-
- `no-read` (no repository read access; `openai-compatible` today): Neal
|
|
193
|
-
inlines the review context — the full diff, plan document content, review
|
|
194
|
-
history, and run artifacts, with per-section truncation caps — directly
|
|
195
|
-
into reviewer prompts, and the doctrine never instructs repository, file,
|
|
196
|
-
tool, or shell access.
|
|
197
182
|
- `tool-access` (read and shell access): a reviewer is instructed to inspect
|
|
198
183
|
and execute against the repository directly. Because the read-only reviewer
|
|
199
184
|
invariant forbids any supported `structured-advisor` capability from holding
|
|
200
185
|
shell access (see
|
|
201
|
-
[The
|
|
202
|
-
built-in reviewer resolves to this mode today
|
|
186
|
+
[The read-only reviewer invariant](#the-read-only-reviewer-invariant)), no
|
|
187
|
+
built-in reviewer resolves to this mode today. It would require a
|
|
203
188
|
shell-capable reviewer, which the registry assertion rejects.
|
|
204
189
|
|
|
205
|
-
|
|
206
|
-
|
|
207
|
-
|
|
208
|
-
|
|
209
|
-
|
|
190
|
+
Read-tool reviewers are never instructed to run verification: the doctrine
|
|
191
|
+
reflects that verification is the coder's, trusted by the reviewer's reading.
|
|
192
|
+
Write access does not affect review doctrine: reviews never instruct repository
|
|
193
|
+
mutation, and no supported reviewer capability declares write access in any
|
|
194
|
+
case.
|
|
210
195
|
|
|
211
|
-
## Runtime
|
|
196
|
+
## Runtime turns
|
|
212
197
|
|
|
213
|
-
Provider adapters receive
|
|
198
|
+
Provider adapters receive neal-owned turn arguments rather than reading neal
|
|
214
199
|
configuration directly.
|
|
215
200
|
|
|
216
201
|
Plain coder turns receive:
|
|
@@ -220,7 +205,7 @@ Plain coder turns receive:
|
|
|
220
205
|
- `inactivityTimeoutMs`
|
|
221
206
|
- optional `resumeHandle`
|
|
222
207
|
- optional structured `outputSchema`, retained only for raw coder prompt
|
|
223
|
-
compatibility outside
|
|
208
|
+
compatibility outside neal product control paths
|
|
224
209
|
- optional `onSessionStarted` callback
|
|
225
210
|
- optional provider event sink
|
|
226
211
|
|
|
@@ -257,9 +242,9 @@ All turn argument shapes also accept an optional `signal` (`AbortSignal`).
|
|
|
257
242
|
Adapters must treat an aborted signal as a request to terminate the in-flight
|
|
258
243
|
SDK turn and surface a normalized provider error rather than hanging.
|
|
259
244
|
|
|
260
|
-
### Turn
|
|
245
|
+
### Turn liveness
|
|
261
246
|
|
|
262
|
-
|
|
247
|
+
neal layers two independent timeouts over every provider turn:
|
|
263
248
|
|
|
264
249
|
- `neal.agent_turn_startup_timeout_ms` (default `300000`) bounds startup
|
|
265
250
|
silence: the window after a turn starts during which the provider has shown
|
|
@@ -271,15 +256,15 @@ Neal layers two independent timeouts over every provider turn:
|
|
|
271
256
|
progress, the startup timer disarms permanently and only this timeout
|
|
272
257
|
applies.
|
|
273
258
|
|
|
274
|
-
When the startup timer fires,
|
|
259
|
+
When the startup timer fires, neal aborts the silent attempt via the turn's
|
|
275
260
|
`signal` and retries the same turn up to `neal.agent_turn_retry_limit` times
|
|
276
|
-
(default `1`). This means
|
|
261
|
+
(default `1`). This means neal may rerun a turn even though the provider never
|
|
277
262
|
reported an error: the provider went silent before doing any work, which is
|
|
278
263
|
indistinguishable from a stalled SDK stream. Retries are safe because zero
|
|
279
264
|
observable progress was made. Keep `agent_turn_retry_limit` low (the default of
|
|
280
|
-
1 is recommended)
|
|
265
|
+
1 is recommended). Persistent startup silence usually indicates a provider or
|
|
281
266
|
network problem that more retries will not fix. Coder turns that resume an
|
|
282
|
-
existing session are never retried by this layer
|
|
267
|
+
existing session are never retried by this layer. The orchestrator's existing
|
|
283
268
|
fresh-session recovery owns those.
|
|
284
269
|
|
|
285
270
|
Liveness outcomes are auditable in `events.ndjson` as
|
|
@@ -294,48 +279,49 @@ Provider session handles are opaque strings. Providers should preserve and
|
|
|
294
279
|
return the SDK session handle when one exists, including on normalized failures
|
|
295
280
|
where the SDK exposed a session before the failure.
|
|
296
281
|
|
|
297
|
-
|
|
282
|
+
neal persists handles separately for planner, coder, and reviewer work. Resume
|
|
298
283
|
checks fail early if the configured provider role does not support session
|
|
299
284
|
resume and a persisted handle must be resumed.
|
|
300
285
|
|
|
301
|
-
## Structured
|
|
286
|
+
## Structured output
|
|
302
287
|
|
|
303
|
-
|
|
288
|
+
neal product control paths use `neal-json-block-v1`. Provider adapters transport
|
|
304
289
|
assistant text, preserve session handles, normalize SDK errors, and delegate
|
|
305
290
|
control-payload extraction, parsing, validation, bounded repair, telemetry, and
|
|
306
|
-
normalized structured-output errors to
|
|
307
|
-
`
|
|
308
|
-
spec (schema, validator,
|
|
309
|
-
|
|
310
|
-
|
|
311
|
-
|
|
312
|
-
|
|
291
|
+
normalized structured-output errors to neal's shared runtime. The
|
|
292
|
+
`openai-compatible` adapter is the one exception: its coder and
|
|
293
|
+
structured-advisor paths consume the same protocol spec (schema, validator,
|
|
294
|
+
labels) but fulfill it through the AI SDK's native structured-output channel in
|
|
295
|
+
a dedicated finalization turn, with no fenced-text transport. See
|
|
296
|
+
[OpenAI-compatible provider](#openai-compatible-provider).
|
|
297
|
+
|
|
298
|
+
Coder adapters expose `runStructuredPrompt<TStructured>()` for neal-owned
|
|
313
299
|
structured coder decisions. That method receives a caller-owned
|
|
314
300
|
`neal-json-block-v1` protocol spec, returns the typed `structured` object plus
|
|
315
301
|
the provider `sessionHandle`, emits `structured_output_received` only after
|
|
316
|
-
|
|
302
|
+
neal validation succeeds, and normalizes missing structured control payloads as
|
|
317
303
|
`structured_output_missing`.
|
|
318
304
|
|
|
319
305
|
`runPrompt(..., outputSchema)` remains supported for compatibility with
|
|
320
|
-
older raw coder harness coverage. `neal check` and
|
|
306
|
+
older raw coder harness coverage. `neal check` and neal-owned coder decision
|
|
321
307
|
rounds use `neal-json-block-v1` through `runStructuredPrompt()` rather than
|
|
322
308
|
parsing provider JSON strings in round code.
|
|
323
309
|
|
|
324
|
-
|
|
310
|
+
neal's shared `neal-json-block-v1` runtime appends provider-neutral transport
|
|
325
311
|
instructions requiring optional useful prose followed by exactly one final
|
|
326
312
|
fenced `neal-json` JSON block. It then extracts, parses, validates, and repairs
|
|
327
313
|
that control object with the caller-supplied schema label, schema, validator,
|
|
328
314
|
and repair limit. Raw whole-response JSON objects are accepted only as a
|
|
329
|
-
compatibility tolerance for older mocks and pre-migration paths
|
|
315
|
+
compatibility tolerance for older mocks and pre-migration paths. The prompt
|
|
330
316
|
contract remains prose plus one final `neal-json` block. State-facing
|
|
331
|
-
`structured_output_received` telemetry is emitted only after
|
|
317
|
+
`structured_output_received` telemetry is emitted only after neal validation
|
|
332
318
|
succeeds.
|
|
333
319
|
|
|
334
320
|
Anthropic Claude and OpenAI Codex structured coder and structured-advisor
|
|
335
321
|
protocol calls do not pass SDK-native `outputFormat` or `outputSchema`. Those
|
|
336
|
-
SDK-native surfaces are compatibility details for non-
|
|
337
|
-
not the
|
|
338
|
-
a bounded repair turn. Repair turns are side-effect-free by prompt
|
|
322
|
+
SDK-native surfaces are compatibility details for non-neal raw provider calls,
|
|
323
|
+
not the neal product protocol. Invalid or missing local JSON can recover through
|
|
324
|
+
a bounded repair turn. Repair turns are side-effect-free by prompt. Claude
|
|
339
325
|
repair turns also omit repo tools and session resume, and Codex repair turns run
|
|
340
326
|
through the most restrictive available streamed-turn options.
|
|
341
327
|
|
|
@@ -374,8 +360,8 @@ round code should not import SDK-specific provider error classes.
|
|
|
374
360
|
Normalized error kinds are:
|
|
375
361
|
|
|
376
362
|
- `timeout`
|
|
377
|
-
- `no_progress_timeout` (emitted by
|
|
378
|
-
adapters
|
|
363
|
+
- `no_progress_timeout` (emitted by neal's turn liveness supervisor, not by
|
|
364
|
+
adapters, as described under Turn liveness above)
|
|
379
365
|
- `api_error`
|
|
380
366
|
- `structured_output_missing`
|
|
381
367
|
- `structured_output_invalid`
|
|
@@ -388,10 +374,10 @@ Each normalized error carries the provider id, role, optional session handle,
|
|
|
388
374
|
kind, retryability, and cause. Round code wraps normalized provider errors into
|
|
389
375
|
round-level failures only when it needs round-specific context.
|
|
390
376
|
|
|
391
|
-
## Status-
|
|
377
|
+
## Status-visible errors
|
|
392
378
|
|
|
393
379
|
Provider failures are visible to automation through `neal status --json` and
|
|
394
|
-
`neal status --json --all` as `providerError`.
|
|
380
|
+
`neal status --json --all` as `providerError`. neal derives that summary from
|
|
395
381
|
the latest `provider.provider_error` event when present, including timestamp,
|
|
396
382
|
provider id, role, label, session handle, normalized kind, bounded message, and
|
|
397
383
|
retryability. If a run failed before a provider event was available, status may
|
|
@@ -402,21 +388,21 @@ fall back to the latest `phase.error` as an unclassified run error with
|
|
|
402
388
|
must not include SDK payloads, full prompts, provider `providerData`, full
|
|
403
389
|
assistant text, raw command output, credentials, or environment dumps. Keep rich
|
|
404
390
|
diagnostics in local support artifacts such as `events.ndjson` and
|
|
405
|
-
`stderr.log
|
|
391
|
+
`stderr.log`. Public wrappers should use the bounded status summary.
|
|
406
392
|
|
|
407
|
-
## Model
|
|
393
|
+
## Model overrides
|
|
408
394
|
|
|
409
395
|
Provider factories receive an optional model override from `agent.*.model`.
|
|
410
|
-
`model: null` means
|
|
396
|
+
`model: null` means neal lets the provider choose its default model. A non-null
|
|
411
397
|
model is accepted only when the provider role declares `supportsModelOverride`.
|
|
412
398
|
|
|
413
399
|
Model selection stays provider-local after the registry passes the configured
|
|
414
400
|
override into the adapter factory.
|
|
415
401
|
|
|
416
|
-
## Reasoning
|
|
402
|
+
## Reasoning effort overrides
|
|
417
403
|
|
|
418
404
|
Each role accepts an optional reasoning-effort override from `agent.*.effort`.
|
|
419
|
-
`effort: null` (or an omitted key) means
|
|
405
|
+
`effort: null` (or an omitted key) means neal sends no effort option and the
|
|
420
406
|
provider uses its default reasoning depth. A non-null effort is mapped to the
|
|
421
407
|
closest provider-native option: OpenAI Codex receives
|
|
422
408
|
`ThreadOptions.modelReasoningEffort` and Anthropic Claude receives
|
|
@@ -426,10 +412,13 @@ Supported values are provider-specific:
|
|
|
426
412
|
|
|
427
413
|
- `openai-codex`: `minimal`, `low`, `medium`, `high`, `xhigh`.
|
|
428
414
|
- `anthropic-claude`: `low`, `medium`, `high`, `xhigh`, `max`.
|
|
429
|
-
- `openai-compatible`: no supported values
|
|
430
|
-
with the standard effort config error.
|
|
431
|
-
|
|
432
|
-
|
|
415
|
+
- `openai-compatible`: no supported values. Any configured `effort` is rejected
|
|
416
|
+
with the standard effort config error. This is a neal limitation, not a claim
|
|
417
|
+
that compatible endpoints cannot control reasoning. Some expose an effort or
|
|
418
|
+
token-budget setting, but the accepted values and behavior vary by endpoint
|
|
419
|
+
and model. Supporting that safely requires an endpoint-specific pass-through
|
|
420
|
+
or capability contract. Until then, neal leaves reasoning at the upstream
|
|
421
|
+
default.
|
|
433
422
|
|
|
434
423
|
Effort support is declared per capability role. An effort value the configured
|
|
435
424
|
provider does not support is rejected before a writer run starts or resumes with
|
|
@@ -442,20 +431,21 @@ is set explicitly, an omitted `agent.planner.effort` resolves to provider
|
|
|
442
431
|
default (`null`) rather than inheriting the coder effort, so set
|
|
443
432
|
`agent.planner.effort` explicitly to override it.
|
|
444
433
|
|
|
445
|
-
## OpenAI-
|
|
446
|
-
|
|
447
|
-
The `openai-compatible` provider
|
|
448
|
-
OpenAI-compatible Chat Completions endpoint
|
|
449
|
-
|
|
450
|
-
|
|
451
|
-
|
|
452
|
-
|
|
453
|
-
|
|
454
|
-
|
|
455
|
-
|
|
456
|
-
|
|
457
|
-
|
|
458
|
-
|
|
434
|
+
## OpenAI-compatible provider
|
|
435
|
+
|
|
436
|
+
The `openai-compatible` provider runs the planner, coder, and reviewer roles
|
|
437
|
+
against any OpenAI-compatible Chat Completions endpoint with a strong
|
|
438
|
+
tool-calling model (OpenRouter, vLLM, DeepSeek, and similar). The planner uses
|
|
439
|
+
the coder path. neal owns
|
|
440
|
+
the agentic loop (one model turn per request, with a neal-owned outer loop
|
|
441
|
+
over the message history) and supplies its own tool set (`read_file`,
|
|
442
|
+
`write_file`, `edit_file`, `list_dir`, `grep`, and `run`), so any endpoint
|
|
443
|
+
whose model can call tools reliably can serve as the coder, and, through a
|
|
444
|
+
read-only subset of the same tools, as the reviewer.
|
|
445
|
+
|
|
446
|
+
The adapter is implemented on the AI SDK's OpenAI-compatible chat provider.
|
|
447
|
+
Gateway behaviors learned from live failures are preserved through a
|
|
448
|
+
neal-owned response interceptor: upstream errors embedded as an `error`
|
|
459
449
|
object inside an HTTP 200 body keep their status-based retryability
|
|
460
450
|
(OpenRouter wraps upstream 429s this way), and reasoning-model responses
|
|
461
451
|
whose final text arrives in `message.reasoning` / `message.reasoning_content`
|
|
@@ -464,37 +454,91 @@ AI SDK's normalized shape (`inputTokens` / `outputTokens` / `totalTokens`
|
|
|
464
454
|
plus detail fields), with the raw response usage object preserved under
|
|
465
455
|
`usage.raw`.
|
|
466
456
|
|
|
467
|
-
|
|
468
|
-
|
|
469
|
-
|
|
470
|
-
|
|
471
|
-
|
|
472
|
-
|
|
473
|
-
|
|
474
|
-
|
|
475
|
-
|
|
476
|
-
|
|
477
|
-
|
|
478
|
-
|
|
479
|
-
|
|
480
|
-
|
|
481
|
-
|
|
482
|
-
[
|
|
483
|
-
|
|
484
|
-
|
|
485
|
-
-
|
|
486
|
-
|
|
487
|
-
|
|
488
|
-
|
|
489
|
-
-
|
|
490
|
-
|
|
491
|
-
|
|
457
|
+
Because endpoint quality varies, qualify an `openai-compatible` model with
|
|
458
|
+
[`neal compat`](compat.md) before trusting it with real work: it drives the
|
|
459
|
+
candidate through complete neal runs against trivial bundled fixtures and emits
|
|
460
|
+
a binary PASS/FAIL per (model, role). `neal check` proves only connectivity and
|
|
461
|
+
basic protocol. `neal compat` proves the full loop.
|
|
462
|
+
|
|
463
|
+
Role support and behavior:
|
|
464
|
+
|
|
465
|
+
- Coder-first. The coder capability role is read, write, and shell capable.
|
|
466
|
+
The planner inherits the coder provider unless configured explicitly.
|
|
467
|
+
- Tool-capable reviewer. The structured-advisor role runs the same
|
|
468
|
+
neal-owned loop over a read-only toolset (`read_file`, `list_dir`,
|
|
469
|
+
`grep`, and the read-only `git_diff` commit-range inspector), so the
|
|
470
|
+
reviewer inspects the repository directly under the `read-only` review
|
|
471
|
+
doctrine mode (see
|
|
472
|
+
[Review doctrine access modes](#review-doctrine-access-modes)). `git_diff`
|
|
473
|
+
is a neal-owned fixed-argv git query (never a shell string) that gives the
|
|
474
|
+
reviewer actual commit-range visibility, including deletions and renames
|
|
475
|
+
that head-state file reads cannot reveal. After the read-tool loop
|
|
476
|
+
completes with a zero-tool-call turn, the structured payload is requested
|
|
477
|
+
in a single dedicated finalization turn that carries no tools. The same
|
|
478
|
+
adapter also serves the final-completion summary gate when
|
|
479
|
+
`openai-compatible` is the coder.
|
|
480
|
+
- No session resume (`supportsSessionResume: false` on both roles). The
|
|
481
|
+
adapter never persists a provider session handle, so `neal resume` after an
|
|
482
|
+
interruption or failure restarts the interrupted scope from scratch in a
|
|
483
|
+
fresh session rather than resuming mid-conversation. Already-accepted
|
|
484
|
+
scopes stay committed. Only the in-flight scope is redone. Top-level
|
|
485
|
+
plan-refinement revision rounds likewise start a fresh planner session per
|
|
486
|
+
round instead of resuming the planning conversation.
|
|
487
|
+
- Step caps: each coder prompt's tool loop is bounded by the exported
|
|
488
|
+
`OPENAI_COMPATIBLE_MAX_STEPS` constant (currently `48` model turns per
|
|
489
|
+
prompt). Structured-advisor/reviewer rounds use the smaller exported
|
|
490
|
+
`OPENAI_COMPATIBLE_ADVISOR_MAX_STEPS` constant (currently `24`: reviews are
|
|
491
|
+
bounded inspections, not implementations). Reaching either cap fails the
|
|
492
|
+
attempt with a non-retryable `provider_failed` error naming the cap. There
|
|
493
|
+
are no config knobs for the caps.
|
|
494
|
+
- Reviewer telemetry: advisor rounds report cumulative per-tool call and
|
|
495
|
+
error counts plus a `steps` count (model turns consumed) under
|
|
496
|
+
`providerData` on `turn_completed` / `usage_reported` events, so
|
|
497
|
+
tool-turns-per-review is auditable from `events.ndjson` when qualifying a
|
|
498
|
+
model for reviewer duty.
|
|
499
|
+
- No `effort` support. Any configured `effort` for this provider is rejected
|
|
500
|
+
with the standard effort config error. See
|
|
501
|
+
[Reasoning effort overrides](#reasoning-effort-overrides) for why.
|
|
502
|
+
- Structured output is SDK-native. After the tool loop's normal
|
|
503
|
+
zero-tool-call completion turn, the adapter makes exactly one dedicated
|
|
504
|
+
finalization turn (no tools) through the AI SDK's `Output.object`
|
|
505
|
+
structured-output channel and validates the result against the protocol
|
|
506
|
+
schema. There is no fenced-JSON text protocol on this provider and no
|
|
507
|
+
second chance for a malformed payload: a finalization failure is a single
|
|
508
|
+
non-retryable error (kind `structured_output_missing` when the model
|
|
509
|
+
produced no parseable object, or `structured_output_invalid` when the
|
|
510
|
+
object failed validation) with a truncated excerpt of the rejected text
|
|
511
|
+
preserved under the `provider_error` event's `providerData.diagnostic`, so
|
|
512
|
+
failed runs stay classifiable. Tool inputs are likewise strict: arguments
|
|
513
|
+
must match the tool's schema as sent, with no coercion of stringified
|
|
514
|
+
JSON. This failure taxonomy is the qualification signal for candidate
|
|
515
|
+
models: a model that cannot pass `neal check` on this provider is not
|
|
516
|
+
pool-eligible.
|
|
517
|
+
- Transient API failures (408/429/5xx, network errors, and empty
|
|
518
|
+
HTTP-200 responses) are retried with bounded backoff and surfaced as
|
|
519
|
+
`api_retry` progress events.
|
|
520
|
+
|
|
521
|
+
Trust level is identical to the vendor writer providers: the `run` tool
|
|
522
|
+
executes arbitrary shell commands in the scope working tree with no sandbox.
|
|
523
|
+
The path jail on file-tool arguments keeps `read_file`, `write_file`,
|
|
524
|
+
`edit_file`, `list_dir`, and `grep` inside the repository (including through
|
|
525
|
+
symlinks), but it does not contain what `run` can do. Apply the same
|
|
526
|
+
disposable-checkout guidance as for the other writer providers.
|
|
527
|
+
Structured-advisor rounds never receive `write_file`, `edit_file`, or `run`.
|
|
528
|
+
The read-only toolset is jailed the same way.
|
|
529
|
+
|
|
530
|
+
Model guidance: use a paid tool-calling slug for coder duty. Rate-limited
|
|
531
|
+
`:free` pools are not suitable as coders, and they are usually too flaky for
|
|
532
|
+
reviewer duty. Treat the dated [compatible-models.md](compatible-models.md)
|
|
533
|
+
whitelist as the current evidence for `openai-compatible` model compatibility.
|
|
534
|
+
It is a contract smoke test, not a skill ranking, so still trial unfamiliar
|
|
535
|
+
models on a disposable project before using them on real work.
|
|
492
536
|
|
|
493
537
|
### Configuration
|
|
494
538
|
|
|
495
|
-
Add a `providers.openai_compatible` block (repo `
|
|
496
|
-
`~/.neal/config.yml`, matching normal config precedence) and point
|
|
497
|
-
|
|
539
|
+
Add a `providers.openai_compatible` block (repo `neal.yml` overrides
|
|
540
|
+
`~/.neal/config.yml`, matching normal config precedence) and point one or
|
|
541
|
+
more roles at the provider. A worked DeepSeek example:
|
|
498
542
|
|
|
499
543
|
```yaml
|
|
500
544
|
providers:
|
|
@@ -512,23 +556,46 @@ agent:
|
|
|
512
556
|
model: deepseek-chat
|
|
513
557
|
```
|
|
514
558
|
|
|
559
|
+
A worked OpenRouter example driving the coder through the provider:
|
|
560
|
+
|
|
561
|
+
```yaml
|
|
562
|
+
providers:
|
|
563
|
+
openai_compatible:
|
|
564
|
+
base_url: https://openrouter.ai/api/v1
|
|
565
|
+
api_key_env: OPENROUTER_API_KEY
|
|
566
|
+
default_model: qwen/qwen3-coder
|
|
567
|
+
|
|
568
|
+
agent:
|
|
569
|
+
coder:
|
|
570
|
+
provider: openai-compatible
|
|
571
|
+
model: null
|
|
572
|
+
reviewer:
|
|
573
|
+
provider: anthropic-claude
|
|
574
|
+
model: null
|
|
575
|
+
```
|
|
576
|
+
|
|
577
|
+
The planner inherits the coder provider unless configured explicitly, so the
|
|
578
|
+
OpenRouter config above routes planning through `openai-compatible` too.
|
|
579
|
+
Structured-advisor rounds additionally honor a neal-internal round-level model
|
|
580
|
+
override first.
|
|
581
|
+
|
|
515
582
|
Settings resolve config-first with environment fallbacks:
|
|
516
583
|
|
|
517
584
|
- `base_url`: `providers.openai_compatible.base_url`, else
|
|
518
585
|
`OPENAI_COMPATIBLE_BASE_URL`. Required.
|
|
519
586
|
- `api_key_env`: `providers.openai_compatible.api_key_env`, else the default
|
|
520
587
|
`OPENAI_COMPATIBLE_API_KEY`. The API key value is read from that named
|
|
521
|
-
environment variable
|
|
522
|
-
- model: the role-level `agent
|
|
588
|
+
environment variable. neal never stores the key itself.
|
|
589
|
+
- model: the role-level `agent.<role>.model` override, else
|
|
523
590
|
`providers.openai_compatible.default_model`, else `OPENAI_COMPATIBLE_MODEL`.
|
|
524
591
|
One of these is required.
|
|
525
592
|
- `headers`: optional string-to-string map of extra HTTP headers (useful for
|
|
526
593
|
OpenRouter attribution headers).
|
|
527
594
|
- `pricing`: an **optional override** for per-million-token rates. It is no
|
|
528
|
-
longer required to get a dollar cost (see "Cost pricing" below)
|
|
595
|
+
longer required to get a dollar cost (see "Cost pricing" below). Set it only
|
|
529
596
|
for models the vendored rate card does not key exactly, or to pin different
|
|
530
597
|
rates. All three rates are required when the block is present (a partial block
|
|
531
|
-
is a configuration error)
|
|
598
|
+
is a configuration error). Rates are in USD per one million tokens:
|
|
532
599
|
|
|
533
600
|
```yaml
|
|
534
601
|
providers:
|
|
@@ -544,32 +611,31 @@ Settings resolve config-first with environment fallbacks:
|
|
|
544
611
|
|
|
545
612
|
#### Cost pricing
|
|
546
613
|
|
|
547
|
-
|
|
548
|
-
|
|
549
|
-
1. **Provider-reported cost
|
|
550
|
-
own `total_cost_usd
|
|
551
|
-
2. **Configured `pricing` override
|
|
552
|
-
`providers.openai_compatible.pricing` block above
|
|
553
|
-
|
|
554
|
-
|
|
555
|
-
|
|
556
|
-
|
|
557
|
-
(retrieved 2026-07-15), so any adapter whose resolved model slug is exactly a
|
|
614
|
+
neal resolves each run's dollar cost per provider/role bucket in this order:
|
|
615
|
+
|
|
616
|
+
1. **Provider-reported cost.** The Claude adapter passes through the provider's
|
|
617
|
+
own `total_cost_usd`. It always wins when present.
|
|
618
|
+
2. **Configured `pricing` override.** The
|
|
619
|
+
`providers.openai_compatible.pricing` block above.
|
|
620
|
+
3. **Vendored published rate card.** The default rate source, keyed by model
|
|
621
|
+
slug. neal ships a trimmed copy of LiteLLM's
|
|
622
|
+
[`model_prices_and_context_window.json`](https://raw.githubusercontent.com/BerriAI/litellm/d9661222492a098555f40cb8b50014054bea5ab8/model_prices_and_context_window.json)
|
|
623
|
+
(retrieved 2026-07-18), so any adapter whose resolved model slug is exactly a
|
|
558
624
|
card key gets a rate-computed cost with **zero configuration**.
|
|
559
|
-
4. **Tokens only
|
|
560
|
-
counts only.
|
|
625
|
+
4. **Tokens only.** If none of the above yields pricing, the run shows token
|
|
626
|
+
counts only. neal never invents dollars.
|
|
561
627
|
|
|
562
628
|
Card lookup is **exact-match only**: the resolved model slug must be an exact
|
|
563
|
-
card key.
|
|
629
|
+
card key. neal does not strip provider prefixes and does not fall back to the
|
|
564
630
|
basename after a `/`, so a slash-qualified slug (for example a local or gateway
|
|
565
631
|
slug like `local/gpt-5.5` or `azure/<deployment>`) is priced only if the card
|
|
566
|
-
lists that exact string
|
|
632
|
+
lists that exact string. Otherwise it stays tokens-only. LiteLLM lists many
|
|
567
633
|
`vendor/model` keys directly, so provider-qualified slugs are still priced when
|
|
568
634
|
the card carries that exact key. Set an explicit `pricing` override for slugs the
|
|
569
635
|
card does not key exactly.
|
|
570
636
|
|
|
571
637
|
**Base-tier only.** Card cost uses each model's published base /
|
|
572
|
-
standard-context per-token rates only.
|
|
638
|
+
standard-context per-token rates only. neal does **not** apply long-context
|
|
573
639
|
surcharges, tiered, batch, or priority rates. A turn whose prompt crosses a
|
|
574
640
|
model's long-context threshold (for example GPT-5.6 Sol above 272K prompt tokens,
|
|
575
641
|
which upstream prices at 2× input / 1.5× output) is priced at the base tier and
|
|
@@ -577,27 +643,27 @@ is therefore an **underestimate** for that turn. Operators who need exact
|
|
|
577
643
|
long-context cost can pin explicit rates via the `pricing` override.
|
|
578
644
|
|
|
579
645
|
The `openai-codex` provider is now card-priced by its configured model (each
|
|
580
|
-
role is priced by the model it actually ran)
|
|
646
|
+
role is priced by the model it actually ran). There is no codex pricing config
|
|
581
647
|
block by design, so a codex role with no configured model (SDK default) stays
|
|
582
648
|
tokens-only.
|
|
583
649
|
|
|
584
650
|
The card is community-maintained list prices, and rate-computed cost is an
|
|
585
651
|
estimate. Rate-computed cost (from either the vendored card or a configured
|
|
586
652
|
override) is flagged with a footnote (`*`) in the retrospective's Provider Usage
|
|
587
|
-
table
|
|
588
|
-
provider."
|
|
589
|
-
input is billed once, at the cached rate.
|
|
653
|
+
table. The note reads "Cost estimated from published or configured rates, not
|
|
654
|
+
reported by the provider." That distinguishes it from cost a provider reports
|
|
655
|
+
directly. Cached input is billed once, at the cached rate.
|
|
590
656
|
|
|
591
|
-
`neal setup` offers `openai-compatible` for the
|
|
592
|
-
guidance when the base URL, API key, or model is unresolved.
|
|
593
|
-
detection reports which config keys or environment variables are
|
|
594
|
-
without making network calls.
|
|
657
|
+
`neal setup` offers `openai-compatible` for the planner, coder, and reviewer
|
|
658
|
+
roles and prints guidance when the base URL, API key, or model is unresolved.
|
|
659
|
+
`neal setup` detection reports which config keys or environment variables are
|
|
660
|
+
missing without making network calls.
|
|
595
661
|
|
|
596
|
-
### Live
|
|
662
|
+
### Live smoke check (optional)
|
|
597
663
|
|
|
598
664
|
The default test suite never requires live credentials or network access. To
|
|
599
665
|
verify a real endpoint end to end, set credentials and run `neal check` with
|
|
600
|
-
the reviewer role configured for `openai-compatible
|
|
666
|
+
the reviewer role configured for `openai-compatible`. The `provider-check`
|
|
601
667
|
structured round exercises the adapter against the live endpoint:
|
|
602
668
|
|
|
603
669
|
```sh
|
|
@@ -607,147 +673,14 @@ OPENAI_COMPATIBLE_MODEL=deepseek-chat \
|
|
|
607
673
|
pnpm start -- check
|
|
608
674
|
```
|
|
609
675
|
|
|
610
|
-
|
|
611
|
-
|
|
612
|
-
The `generic-agentic` provider runs the coder role (and the inherited planner
|
|
613
|
-
role) against any OpenAI-compatible Chat Completions endpoint with a strong
|
|
614
|
-
tool-calling model (OpenRouter, vLLM, DeepSeek, and similar). Unlike the
|
|
615
|
-
API-only `openai-compatible` reviewer adapter, this provider is a real writer:
|
|
616
|
-
Neal owns the agentic loop (one model turn per request, with a Neal-owned
|
|
617
|
-
outer loop over the message history) and supplies its own tool set —
|
|
618
|
-
`read_file`, `write_file`, `edit_file`, `list_dir`, `grep`, and `run` — so any
|
|
619
|
-
endpoint whose model can call tools reliably can serve as the coder, and,
|
|
620
|
-
through a read-only subset of the same tools, as the reviewer.
|
|
676
|
+
### Local endpoints (Ollama, llama.cpp, vLLM)
|
|
621
677
|
|
|
622
|
-
|
|
623
|
-
|
|
624
|
-
candidate through complete neal runs against trivial bundled fixtures and emits
|
|
625
|
-
a binary PASS/FAIL per (model, role). `neal check` proves only connectivity and
|
|
626
|
-
basic protocol; `neal compat` proves the full loop.
|
|
627
|
-
|
|
628
|
-
Role support and behavior:
|
|
629
|
-
|
|
630
|
-
- Coder-first. The coder capability role is read, write, and shell capable;
|
|
631
|
-
the planner inherits the coder provider unless configured explicitly.
|
|
632
|
-
- Tool-capable reviewer. The structured-advisor role runs the same
|
|
633
|
-
Neal-owned loop over a read-only toolset — `read_file`, `list_dir`,
|
|
634
|
-
`grep`, and the read-only `git_diff` commit-range inspector — so a generic
|
|
635
|
-
reviewer inspects the repository directly under the `read-only` review
|
|
636
|
-
doctrine mode (see
|
|
637
|
-
[Review Doctrine Access Modes](#review-doctrine-access-modes)) instead of
|
|
638
|
-
receiving Neal-inlined context. `git_diff` is a Neal-owned fixed-argv git
|
|
639
|
-
query (never a shell string) that gives the reviewer actual commit-range
|
|
640
|
-
visibility, including deletions and renames that head-state file reads
|
|
641
|
-
cannot reveal. After the read-tool loop completes with a zero-tool-call
|
|
642
|
-
turn, the structured payload is requested in a single dedicated
|
|
643
|
-
finalization turn that carries no tools. The same adapter also serves the
|
|
644
|
-
final-completion summary gate when `generic-agentic` is the coder.
|
|
645
|
-
- No session resume (`supportsSessionResume: false` on both roles). The
|
|
646
|
-
adapter never persists a provider session handle, so `neal resume` after an
|
|
647
|
-
interruption or failure restarts the interrupted scope from scratch in a
|
|
648
|
-
fresh session rather than resuming mid-conversation. Already-accepted
|
|
649
|
-
scopes stay committed; only the in-flight scope is redone. Top-level
|
|
650
|
-
plan-refinement revision rounds likewise start a fresh planner session per
|
|
651
|
-
round instead of resuming the planning conversation.
|
|
652
|
-
- Step caps: each coder prompt's tool loop is bounded by the exported
|
|
653
|
-
`GENERIC_AGENTIC_MAX_STEPS` constant (currently `48` model turns per
|
|
654
|
-
prompt); structured-advisor/reviewer rounds use the smaller exported
|
|
655
|
-
`GENERIC_AGENTIC_ADVISOR_MAX_STEPS` constant (currently `24` — reviews are
|
|
656
|
-
bounded inspections, not implementations). Reaching either cap fails the
|
|
657
|
-
attempt with a non-retryable `provider_failed` error naming the cap. There
|
|
658
|
-
are no config knobs for the caps.
|
|
659
|
-
- Reviewer telemetry: advisor rounds report cumulative per-tool call and
|
|
660
|
-
error counts plus a `steps` count (model turns consumed) under
|
|
661
|
-
`providerData` on `turn_completed` / `usage_reported` events, so
|
|
662
|
-
tool-turns-per-review is auditable from `events.ndjson` when qualifying a
|
|
663
|
-
model for reviewer duty.
|
|
664
|
-
- No `effort` support. Any configured `effort` for this provider is rejected
|
|
665
|
-
with the standard effort config error.
|
|
666
|
-
- Structured output is SDK-native. After the tool loop's normal
|
|
667
|
-
zero-tool-call completion turn, the adapter makes exactly one dedicated
|
|
668
|
-
finalization turn (no tools) through the AI SDK's `Output.object`
|
|
669
|
-
structured-output channel and validates the result against the protocol
|
|
670
|
-
schema. There is no fenced-JSON text protocol on this provider and no
|
|
671
|
-
second chance for a malformed payload: a finalization failure is a single
|
|
672
|
-
non-retryable error — kind `structured_output_missing` when the model
|
|
673
|
-
produced no parseable object, or `structured_output_invalid` when the
|
|
674
|
-
object failed validation — with a truncated excerpt of the rejected text
|
|
675
|
-
preserved under the `provider_error` event's `providerData.diagnostic`, so
|
|
676
|
-
failed runs stay classifiable. Tool inputs are likewise strict: arguments
|
|
677
|
-
must match the tool's schema as sent, with no coercion of stringified
|
|
678
|
-
JSON. This failure taxonomy is the qualification signal for candidate
|
|
679
|
-
models — a model that cannot pass `neal check` on this provider is not
|
|
680
|
-
pool-eligible.
|
|
681
|
-
- Transient API failures (408/429/5xx, network errors, and empty
|
|
682
|
-
HTTP-200 responses) are retried with bounded backoff and surfaced as
|
|
683
|
-
`api_retry` progress events, matching the `openai-compatible` adapter's
|
|
684
|
-
behavior.
|
|
685
|
-
|
|
686
|
-
Trust level is identical to the vendor writer providers: the `run` tool
|
|
687
|
-
executes arbitrary shell commands in the scope working tree with no sandbox.
|
|
688
|
-
The path jail on file-tool arguments keeps `read_file`, `write_file`,
|
|
689
|
-
`edit_file`, `list_dir`, and `grep` inside the repository (including through
|
|
690
|
-
symlinks), but it does not contain what `run` can do. Apply the same
|
|
691
|
-
disposable-checkout guidance as for the other writer providers.
|
|
692
|
-
Structured-advisor rounds never receive `write_file`, `edit_file`, or `run`;
|
|
693
|
-
the read-only toolset is jailed the same way.
|
|
694
|
-
|
|
695
|
-
Choosing a reviewer adapter for generic endpoints: prefer `generic-agentic`
|
|
696
|
-
when the endpoint's model calls tools reliably — the reviewer reads the
|
|
697
|
-
actual repository and commit range with bounded, targeted tool calls instead
|
|
698
|
-
of judging a truncation-capped inlined diff, and reviewer prompts stay small.
|
|
699
|
-
Prefer `openai-compatible` for genuinely chat-only endpoints or models whose
|
|
700
|
-
tool calling is unreliable; the inline-context path remains correct and fully
|
|
701
|
-
supported.
|
|
702
|
-
|
|
703
|
-
Model guidance: use a paid tool-calling slug for coder duty. Rate-limited
|
|
704
|
-
`:free` pools are acceptable for reviewer-only use but are not suitable as
|
|
705
|
-
coders. Model qualification runs and a tested-models table are future work;
|
|
706
|
-
until that lands, treat models other than `qwen/qwen3-coder` (validated
|
|
707
|
-
end-to-end as the coder) and `openai/gpt-oss-120b` (validated live as the
|
|
708
|
-
read-only reviewer and as the inline-context reviewer) as unqualified and
|
|
709
|
-
trial them on a disposable project first.
|
|
710
|
-
|
|
711
|
-
### Configuration
|
|
712
|
-
|
|
713
|
-
Settings reuse the `providers.openai_compatible` block — there is no separate
|
|
714
|
-
`providers.generic_agentic` config surface — with the same config-first,
|
|
715
|
-
environment-fallback resolution documented in
|
|
716
|
-
[OpenAI-Compatible Endpoints](#openai-compatible-endpoints). A worked
|
|
717
|
-
OpenRouter example:
|
|
718
|
-
|
|
719
|
-
```yaml
|
|
720
|
-
providers:
|
|
721
|
-
openai_compatible:
|
|
722
|
-
base_url: https://openrouter.ai/api/v1
|
|
723
|
-
api_key_env: OPENROUTER_API_KEY
|
|
724
|
-
default_model: qwen/qwen3-coder
|
|
725
|
-
|
|
726
|
-
agent:
|
|
727
|
-
coder:
|
|
728
|
-
provider: generic-agentic
|
|
729
|
-
model: null
|
|
730
|
-
reviewer:
|
|
731
|
-
provider: anthropic-claude
|
|
732
|
-
model: null
|
|
733
|
-
```
|
|
734
|
-
|
|
735
|
-
The planner inherits the coder provider unless configured explicitly, so this
|
|
736
|
-
config routes planning through `generic-agentic` too. Coder model precedence
|
|
737
|
-
is `agent.coder.model`, else `providers.openai_compatible.default_model`,
|
|
738
|
-
else `OPENAI_COMPATIBLE_MODEL`; one of these is required. Structured-advisor
|
|
739
|
-
rounds additionally honor a Neal-internal round-level model override first,
|
|
740
|
-
matching the `openai-compatible` adapter.
|
|
741
|
-
|
|
742
|
-
### Local Endpoints (Ollama, llama.cpp, vLLM)
|
|
743
|
-
|
|
744
|
-
A local server that speaks the OpenAI-compatible Chat Completions API — Ollama,
|
|
745
|
-
llama.cpp's `server`, vLLM, and similar — is just another endpoint for this
|
|
678
|
+
A local server that speaks the OpenAI-compatible Chat Completions API (Ollama,
|
|
679
|
+
llama.cpp's `server`, vLLM, and similar) is just another endpoint for this
|
|
746
680
|
provider. It reuses the same `providers.openai_compatible` block documented
|
|
747
|
-
above
|
|
748
|
-
|
|
749
|
-
|
|
750
|
-
`base_url`, model slug, and auth variable change.
|
|
681
|
+
above, with the same config-first, environment-fallback resolution rules from
|
|
682
|
+
[Configuration](#configuration). Only the `base_url`, model slug, and auth
|
|
683
|
+
variable change.
|
|
751
684
|
|
|
752
685
|
```yaml
|
|
753
686
|
providers:
|
|
@@ -761,49 +694,48 @@ agent:
|
|
|
761
694
|
provider: openai-codex
|
|
762
695
|
model: null
|
|
763
696
|
reviewer:
|
|
764
|
-
provider:
|
|
697
|
+
provider: openai-compatible
|
|
765
698
|
model: null
|
|
766
699
|
```
|
|
767
700
|
|
|
768
|
-
|
|
701
|
+
neal has no `api_key` config field: it resolves the key from the environment
|
|
769
702
|
variable named by `api_key_env` (default `OPENAI_COMPATIBLE_API_KEY`), and the
|
|
770
703
|
AI SDK requires that value to be non-empty even when the local server ignores
|
|
771
|
-
auth. So export a non-empty placeholder to the variable you named
|
|
772
|
-
`export OLLAMA_API_KEY=ollama`
|
|
704
|
+
auth. So export a non-empty placeholder to the variable you named
|
|
705
|
+
(`export OLLAMA_API_KEY=ollama`) before starting a run. Use the fully qualified
|
|
773
706
|
local slug your server reports (Ollama tags such as `qwen3-coder:30b`), not a
|
|
774
707
|
bare family name.
|
|
775
708
|
|
|
776
709
|
Qualify a local model with [`neal compat`](compat.md)
|
|
777
|
-
(`neal compat --model <slug> --role all`) before trusting it with writer runs
|
|
778
|
-
`--model` forces the candidate onto `
|
|
710
|
+
(`neal compat --model <slug> --role all`) before trusting it with writer runs.
|
|
711
|
+
`--model` forces the candidate onto `openai-compatible`. Note the reference
|
|
779
712
|
caveat: `--model` defaults `--reference` to `openai-codex`, which needs a Codex
|
|
780
713
|
login, so a purely-local operator should pass
|
|
781
|
-
`--reference
|
|
714
|
+
`--reference openai-compatible:<same-slug>` for a self-contained check, while
|
|
782
715
|
being aware that self-reference is a weaker qualification partner (a candidate
|
|
783
716
|
grading itself), so treat its PASS with more caution than a native reference
|
|
784
717
|
(see [compatible-models.md](compatible-models.md)).
|
|
785
718
|
|
|
786
719
|
Set realistic role expectations. Local models below the whitelist bar usually
|
|
787
|
-
fail the **coder** role on structured output
|
|
788
|
-
coder-FAIL rows in [compatible-models.md](compatible-models.md)
|
|
720
|
+
fail the **coder** role on structured output (`structured_output` dominates the
|
|
721
|
+
coder-FAIL rows in [compatible-models.md](compatible-models.md)), so the config
|
|
789
722
|
above is the realistic starting point: keep the coder on a native provider
|
|
790
|
-
(`openai-codex` or `anthropic-claude`) and give the local `
|
|
723
|
+
(`openai-codex` or `anthropic-claude`) and give the local `openai-compatible`
|
|
791
724
|
endpoint reviewer-only duty. That split only works if the local model calls
|
|
792
|
-
tools reliably
|
|
793
|
-
|
|
794
|
-
endpoints" guidance above.
|
|
725
|
+
tools reliably: the reviewer path is tool-driven, so a chat-only model (no
|
|
726
|
+
tool calling) is not supported in any role.
|
|
795
727
|
|
|
796
|
-
## Adding
|
|
728
|
+
## Adding a built-in provider
|
|
797
729
|
|
|
798
730
|
To add another built-in provider:
|
|
799
731
|
|
|
800
732
|
1. Implement a provider module under `src/neal/providers/`.
|
|
801
733
|
2. Export a `NealProviderDefinition`.
|
|
802
734
|
3. Register the definition in `src/neal/providers/registry.ts`.
|
|
803
|
-
4. Map SDK sessions and resume handles to opaque
|
|
735
|
+
4. Map SDK sessions and resume handles to opaque neal handles.
|
|
804
736
|
5. Map SDK stream or message events to `ProviderRuntimeEvent` values.
|
|
805
737
|
6. Normalize SDK errors to `NealProviderError`.
|
|
806
|
-
7. Add focused tests for event mapping, capabilities,
|
|
738
|
+
7. Add focused tests for event mapping, capabilities, neal structured control
|
|
807
739
|
protocol behavior, session handles, and normalized errors.
|
|
808
740
|
8. Document SDK prerequisites, environment assumptions, supported roles,
|
|
809
741
|
capabilities, and limitations.
|
|
@@ -813,7 +745,7 @@ not need a real SDK. SDK adapter tests should stay focused on provider-specific
|
|
|
813
745
|
event collection, structured-output parsing, retry behavior, and error
|
|
814
746
|
normalization.
|
|
815
747
|
|
|
816
|
-
## SDK
|
|
748
|
+
## SDK prerequisites
|
|
817
749
|
|
|
818
750
|
Provider documentation should state the SDK package, required credentials or
|
|
819
751
|
login state, model assumptions, filesystem/tool permission assumptions, and any
|
|
@@ -824,18 +756,16 @@ Current built-in writer providers run with broad local permissions:
|
|
|
824
756
|
- OpenAI Codex uses `approvalPolicy: never` and
|
|
825
757
|
`sandboxMode: danger-full-access`.
|
|
826
758
|
- Anthropic Claude uses `permissionMode: bypassPermissions`.
|
|
827
|
-
- OpenAI-
|
|
828
|
-
|
|
829
|
-
|
|
830
|
-
|
|
831
|
-
sandbox, matching the vendor writers' trust level. Structured-advisor
|
|
832
|
-
rounds execute only the read-only toolset locally.
|
|
759
|
+
- OpenAI-compatible makes HTTP API calls for model turns but executes
|
|
760
|
+
neal-owned tools locally. The coder role's `run` tool executes shell
|
|
761
|
+
commands with no sandbox, matching the vendor writers' trust level.
|
|
762
|
+
Structured-advisor rounds execute only the read-only toolset locally.
|
|
833
763
|
|
|
834
764
|
Those settings are part of current writer-run behavior. Do not narrow or widen
|
|
835
765
|
them as part of provider registration unless a separate product change requires
|
|
836
766
|
it.
|
|
837
767
|
|
|
838
|
-
## Review
|
|
768
|
+
## Review boundary
|
|
839
769
|
|
|
840
770
|
Public `neal review` uses a separate read-only findings loop:
|
|
841
771
|
|
|
@@ -850,16 +780,16 @@ writes isolated artifacts under `.neal/reviews/<review-id>/`.
|
|
|
850
780
|
Two distinct read-only guarantees apply here, and they should not be conflated:
|
|
851
781
|
|
|
852
782
|
- The provider-capability invariant (see
|
|
853
|
-
[The
|
|
783
|
+
[The read-only reviewer invariant](#the-read-only-reviewer-invariant)):
|
|
854
784
|
every supported `structured-advisor` capability has `write: false` and
|
|
855
785
|
`shell: false`, so a writer-run reviewer/structured-advisor round cannot
|
|
856
|
-
write or run shell by capability
|
|
786
|
+
write or run shell by capability. This holds for both writer runs and
|
|
857
787
|
`neal review`.
|
|
858
788
|
- The `neal review` command's additional artifact-boundary guard: the command
|
|
859
789
|
checks protected writer state and worktree changes after provider rounds and
|
|
860
790
|
fails if anything outside its review artifacts changed. This post-round guard
|
|
861
791
|
is owned by the `neal review` command flow, not by a provider capability, and
|
|
862
|
-
is specific to `neal review
|
|
792
|
+
is specific to `neal review`. A writer run does not perform this artifact
|
|
863
793
|
diff.
|
|
864
794
|
|
|
865
795
|
So the capability invariant guarantees reviewers never write or run shell in
|