coderouter-cli 2.14.0__py3-none-any.whl → 2.15.1__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- coderouter/adapters/anthropic_native.py +106 -1
- coderouter/adapters/base.py +65 -0
- coderouter/adapters/openai_compat.py +60 -0
- coderouter/config/schemas.py +65 -0
- coderouter/doctor.py +48 -1
- coderouter/ingress/anthropic_routes.py +82 -12
- coderouter/ingress/openai_routes.py +31 -2
- coderouter/logging.py +143 -0
- coderouter/metrics/collector.py +40 -0
- coderouter/metrics/prometheus.py +46 -0
- coderouter/routing/__init__.py +8 -0
- coderouter/routing/fallback.py +304 -4
- coderouter/routing/fallback_trace.py +501 -0
- coderouter/state/audit_log.py +4 -0
- coderouter/vscode_init.py +10 -3
- {coderouter_cli-2.14.0.dist-info → coderouter_cli-2.15.1.dist-info}/METADATA +17 -5
- {coderouter_cli-2.14.0.dist-info → coderouter_cli-2.15.1.dist-info}/RECORD +20 -19
- {coderouter_cli-2.14.0.dist-info → coderouter_cli-2.15.1.dist-info}/WHEEL +1 -1
- {coderouter_cli-2.14.0.dist-info → coderouter_cli-2.15.1.dist-info}/entry_points.txt +0 -0
- {coderouter_cli-2.14.0.dist-info → coderouter_cli-2.15.1.dist-info}/licenses/LICENSE +0 -0
|
@@ -0,0 +1,501 @@
|
|
|
1
|
+
"""v2.15.0: request-scoped fallback trace — *why* did the chain move?
|
|
2
|
+
|
|
3
|
+
Before v2.15.0 the engine could tell an operator (via ``try-provider`` /
|
|
4
|
+
``provider-failed`` log lines) that a fallback happened, but nothing tied
|
|
5
|
+
the two together into a single "we left A for B because of R" record, and
|
|
6
|
+
nothing at all reached the *client*. This module is that missing record.
|
|
7
|
+
|
|
8
|
+
Vocabulary
|
|
9
|
+
==========
|
|
10
|
+
|
|
11
|
+
``reason`` already existed in the codebase as a keyword argument on
|
|
12
|
+
:func:`coderouter.routing.capability.log_capability_degraded`
|
|
13
|
+
(``provider-does-not-support`` / ``translation-lossy`` /
|
|
14
|
+
``unsupported-backend``) and on the drift verdict. Those are **degrade**
|
|
15
|
+
reasons: the request stays on the same provider, something is stripped.
|
|
16
|
+
v2.15.0 keeps those strings verbatim (they are re-exported here as
|
|
17
|
+
``REASON_PROVIDER_DOES_NOT_SUPPORT`` etc. so there is one place to read
|
|
18
|
+
the whole vocabulary) and adds the **fallback** reasons — the ones that
|
|
19
|
+
actually move the request to a different provider.
|
|
20
|
+
|
|
21
|
+
Only fallback reasons ever produce a :class:`FallbackHop`. Degrade
|
|
22
|
+
reasons never do, by construction: nothing moved.
|
|
23
|
+
|
|
24
|
+
Fallback reasons split in two families:
|
|
25
|
+
|
|
26
|
+
*Pre-attempt* (the provider was filtered out of the chain during
|
|
27
|
+
:meth:`FallbackEngine._resolve_chain`, so it was never called):
|
|
28
|
+
``paid-gate`` / ``budget-exceeded`` / ``memory-pressure`` /
|
|
29
|
+
``backend-unhealthy`` / ``self-healing-excluded`` / ``unknown-provider``.
|
|
30
|
+
|
|
31
|
+
*Attempt-failure* (the provider was called and did not deliver):
|
|
32
|
+
``timeout`` / ``rate-limit`` / ``auth`` / ``upstream-5xx`` /
|
|
33
|
+
``upstream-4xx`` / ``connection`` / ``upstream-error`` /
|
|
34
|
+
``empty-response`` / ``empty-stream``.
|
|
35
|
+
|
|
36
|
+
Request scoping
|
|
37
|
+
===============
|
|
38
|
+
|
|
39
|
+
The trace lives in a :class:`~contextvars.ContextVar`, the same isolation
|
|
40
|
+
mechanism the drift verdict (``_drift_verdict_ctx``) and the M11 prepared
|
|
41
|
+
dispatch (``_prepared_dispatch_ctx``) already use in
|
|
42
|
+
:mod:`coderouter.routing.fallback`. Starlette runs each request handler in
|
|
43
|
+
its own asyncio task, so one request never sees another's hops, and the
|
|
44
|
+
value cannot go stale across requests (the ContextVar defaults to
|
|
45
|
+
``None``).
|
|
46
|
+
|
|
47
|
+
Two entry points, deliberately different:
|
|
48
|
+
|
|
49
|
+
* :func:`begin_fallback_trace` — installs a *fresh* trace. Called by the
|
|
50
|
+
engine's dispatch entry points. ``keep_existing=True`` preserves a trace
|
|
51
|
+
the ingress's ``apply_context_budget`` pre-pass already started for the
|
|
52
|
+
same request (chain-resolve skips are recorded there, before dispatch).
|
|
53
|
+
* :func:`current_fallback_trace` — read-only lookup. The ingress uses it
|
|
54
|
+
after dispatch to build the ``X-CodeRouter-Fallback-*`` headers and the
|
|
55
|
+
``coderouter_fallback`` SSE metadata event.
|
|
56
|
+
|
|
57
|
+
Ordering / shape contract (what a client can rely on)
|
|
58
|
+
=====================================================
|
|
59
|
+
|
|
60
|
+
For a request that fell back twice::
|
|
61
|
+
|
|
62
|
+
X-CodeRouter-Fallback-From: local
|
|
63
|
+
X-CodeRouter-Fallback-To: openrouter
|
|
64
|
+
X-CodeRouter-Fallback-Reason: timeout,upstream-5xx
|
|
65
|
+
X-CodeRouter-Fallback-Chain: local>ollama>openrouter
|
|
66
|
+
|
|
67
|
+
``Chain`` is the providers in try-order; ``Reason`` is the reason for each
|
|
68
|
+
*departure*, in the same order. ``len(Chain) == len(Reason) + 1`` when the
|
|
69
|
+
chain finally produced an answer, and ``len(Chain) == len(Reason)`` when
|
|
70
|
+
the whole chain was exhausted (there was no provider to hand off to, so
|
|
71
|
+
``X-CodeRouter-Fallback-To`` is omitted). ``From`` is the first provider
|
|
72
|
+
that was abandoned, ``To`` the one that ultimately served.
|
|
73
|
+
|
|
74
|
+
None of these headers are emitted when no fallback happened — the
|
|
75
|
+
zero-fallback path is byte-for-byte unchanged from v2.14.0.
|
|
76
|
+
"""
|
|
77
|
+
|
|
78
|
+
from __future__ import annotations
|
|
79
|
+
|
|
80
|
+
from contextvars import ContextVar
|
|
81
|
+
from dataclasses import dataclass, field
|
|
82
|
+
from typing import TYPE_CHECKING, Any, Final
|
|
83
|
+
|
|
84
|
+
if TYPE_CHECKING:
|
|
85
|
+
from coderouter.adapters.base import AdapterError
|
|
86
|
+
|
|
87
|
+
# ---------------------------------------------------------------------------
|
|
88
|
+
# Canonical reason vocabulary
|
|
89
|
+
# ---------------------------------------------------------------------------
|
|
90
|
+
|
|
91
|
+
# -- pre-attempt (chain-resolve filters; provider was never called) ---------
|
|
92
|
+
REASON_PAID_GATE: Final = "paid-gate"
|
|
93
|
+
REASON_BUDGET_EXCEEDED: Final = "budget-exceeded"
|
|
94
|
+
REASON_MEMORY_PRESSURE: Final = "memory-pressure"
|
|
95
|
+
REASON_BACKEND_UNHEALTHY: Final = "backend-unhealthy"
|
|
96
|
+
REASON_SELF_HEALING_EXCLUDED: Final = "self-healing-excluded"
|
|
97
|
+
REASON_UNKNOWN_PROVIDER: Final = "unknown-provider"
|
|
98
|
+
|
|
99
|
+
# -- attempt failures (provider was called) --------------------------------
|
|
100
|
+
REASON_TIMEOUT: Final = "timeout"
|
|
101
|
+
REASON_RATE_LIMIT: Final = "rate-limit"
|
|
102
|
+
REASON_AUTH: Final = "auth"
|
|
103
|
+
REASON_UPSTREAM_5XX: Final = "upstream-5xx"
|
|
104
|
+
REASON_UPSTREAM_4XX: Final = "upstream-4xx"
|
|
105
|
+
REASON_CONNECTION: Final = "connection"
|
|
106
|
+
REASON_UPSTREAM_ERROR: Final = "upstream-error"
|
|
107
|
+
REASON_EMPTY_RESPONSE: Final = "empty-response"
|
|
108
|
+
REASON_EMPTY_STREAM: Final = "empty-stream"
|
|
109
|
+
# v2.15.0: the upstream SSE stream ended without its protocol terminator —
|
|
110
|
+
# distinct from ``empty-stream`` (nothing at all arrived) and from
|
|
111
|
+
# ``connection`` (the transport itself broke). Produced only when the profile
|
|
112
|
+
# sets ``stream_truncation_action: error``.
|
|
113
|
+
REASON_STREAM_TRUNCATED: Final = "stream-truncated"
|
|
114
|
+
|
|
115
|
+
# -- degrade reasons (pre-existing strings; never produce a hop) -----------
|
|
116
|
+
# Kept here so the whole ``reason`` vocabulary is readable in one place.
|
|
117
|
+
REASON_PROVIDER_DOES_NOT_SUPPORT: Final = "provider-does-not-support"
|
|
118
|
+
REASON_TRANSLATION_LOSSY: Final = "translation-lossy"
|
|
119
|
+
REASON_UNSUPPORTED_BACKEND: Final = "unsupported-backend"
|
|
120
|
+
|
|
121
|
+
PRE_ATTEMPT_REASONS: Final[frozenset[str]] = frozenset(
|
|
122
|
+
{
|
|
123
|
+
REASON_PAID_GATE,
|
|
124
|
+
REASON_BUDGET_EXCEEDED,
|
|
125
|
+
REASON_MEMORY_PRESSURE,
|
|
126
|
+
REASON_BACKEND_UNHEALTHY,
|
|
127
|
+
REASON_SELF_HEALING_EXCLUDED,
|
|
128
|
+
REASON_UNKNOWN_PROVIDER,
|
|
129
|
+
}
|
|
130
|
+
)
|
|
131
|
+
|
|
132
|
+
ATTEMPT_FAILURE_REASONS: Final[frozenset[str]] = frozenset(
|
|
133
|
+
{
|
|
134
|
+
REASON_TIMEOUT,
|
|
135
|
+
REASON_RATE_LIMIT,
|
|
136
|
+
REASON_AUTH,
|
|
137
|
+
REASON_UPSTREAM_5XX,
|
|
138
|
+
REASON_UPSTREAM_4XX,
|
|
139
|
+
REASON_CONNECTION,
|
|
140
|
+
REASON_UPSTREAM_ERROR,
|
|
141
|
+
REASON_EMPTY_RESPONSE,
|
|
142
|
+
REASON_EMPTY_STREAM,
|
|
143
|
+
REASON_STREAM_TRUNCATED,
|
|
144
|
+
}
|
|
145
|
+
)
|
|
146
|
+
|
|
147
|
+
FALLBACK_REASONS: Final[frozenset[str]] = PRE_ATTEMPT_REASONS | ATTEMPT_FAILURE_REASONS
|
|
148
|
+
|
|
149
|
+
DEGRADE_REASONS: Final[frozenset[str]] = frozenset(
|
|
150
|
+
{
|
|
151
|
+
REASON_PROVIDER_DOES_NOT_SUPPORT,
|
|
152
|
+
REASON_TRANSLATION_LOSSY,
|
|
153
|
+
REASON_UNSUPPORTED_BACKEND,
|
|
154
|
+
}
|
|
155
|
+
)
|
|
156
|
+
|
|
157
|
+
|
|
158
|
+
# ---------------------------------------------------------------------------
|
|
159
|
+
# Header names — follow the v2.0-F / v2.0-G convention already in
|
|
160
|
+
# ``ingress/anthropic_routes.py`` (``X-CodeRouter-Context-Budget`` /
|
|
161
|
+
# ``X-CodeRouter-Drift``): ``X-CodeRouter-`` prefix, Train-Case suffix,
|
|
162
|
+
# short scalar value.
|
|
163
|
+
# ---------------------------------------------------------------------------
|
|
164
|
+
|
|
165
|
+
HEADER_FALLBACK_FROM: Final = "X-CodeRouter-Fallback-From"
|
|
166
|
+
HEADER_FALLBACK_TO: Final = "X-CodeRouter-Fallback-To"
|
|
167
|
+
HEADER_FALLBACK_REASON: Final = "X-CodeRouter-Fallback-Reason"
|
|
168
|
+
HEADER_FALLBACK_CHAIN: Final = "X-CodeRouter-Fallback-Chain"
|
|
169
|
+
|
|
170
|
+
# SSE metadata event name for the streaming path. Mirrors the existing
|
|
171
|
+
# ``coderouter_partial`` extension event (v2.0-H) — a client-optional
|
|
172
|
+
# trailing frame, emitted after ``message_stop``, that spec-compliant
|
|
173
|
+
# Anthropic SDKs ignore.
|
|
174
|
+
SSE_FALLBACK_EVENT: Final = "coderouter_fallback"
|
|
175
|
+
|
|
176
|
+
# Provider names and reasons are config identifiers, but a header value has
|
|
177
|
+
# to stay in the printable-ASCII token range no matter what a user typed in
|
|
178
|
+
# their YAML. Anything outside the allowlist collapses to ``_``.
|
|
179
|
+
_SAFE_CHARS: Final[frozenset[str]] = frozenset(
|
|
180
|
+
"abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789._-"
|
|
181
|
+
)
|
|
182
|
+
|
|
183
|
+
|
|
184
|
+
def _sanitize(value: str) -> str:
|
|
185
|
+
"""Reduce ``value`` to a header-safe token.
|
|
186
|
+
|
|
187
|
+
Keeps ``[A-Za-z0-9._-]`` and replaces everything else with ``_`` so a
|
|
188
|
+
provider named ``my provider`` or one carrying non-ASCII can never
|
|
189
|
+
produce a malformed (or header-injecting) response header.
|
|
190
|
+
"""
|
|
191
|
+
return "".join(ch if ch in _SAFE_CHARS else "_" for ch in value) or "_"
|
|
192
|
+
|
|
193
|
+
|
|
194
|
+
def describe_adapter_error(exc: AdapterError) -> str:
|
|
195
|
+
"""Build the short, non-leaking ``detail`` string for a failed attempt.
|
|
196
|
+
|
|
197
|
+
Deliberately structural: the upstream's error *text* stays in the
|
|
198
|
+
existing ``provider-failed`` warn line (which passes through the
|
|
199
|
+
secret-redaction filter) and never reaches a response header or an SSE
|
|
200
|
+
event. Here we only report the HTTP status, or the transport shape
|
|
201
|
+
when there was no response at all.
|
|
202
|
+
"""
|
|
203
|
+
if exc.status_code is not None:
|
|
204
|
+
return f"status={exc.status_code}"
|
|
205
|
+
reason = classify_adapter_error(exc)
|
|
206
|
+
if reason == REASON_STREAM_TRUNCATED:
|
|
207
|
+
# Not a transport fault — the HTTP body ended cleanly; it was the
|
|
208
|
+
# LLM protocol inside it that never closed.
|
|
209
|
+
return "no-terminator"
|
|
210
|
+
return f"transport={reason}"
|
|
211
|
+
|
|
212
|
+
|
|
213
|
+
def classify_adapter_error(exc: AdapterError) -> str:
|
|
214
|
+
"""Map an :class:`AdapterError` onto a canonical fallback reason.
|
|
215
|
+
|
|
216
|
+
Status codes are the primary signal. Adapters raise with
|
|
217
|
+
``status_code=None`` for pre-response failures (httpx timeout,
|
|
218
|
+
transport error, JSON decode), so those are disambiguated by the
|
|
219
|
+
message text the adapters already write — ``timeout ...`` and
|
|
220
|
+
``transport error: ...`` are stable prefixes in both
|
|
221
|
+
``adapters/openai_compat.py`` and ``adapters/anthropic_native.py``.
|
|
222
|
+
|
|
223
|
+
v2.15.0: ``StreamTruncatedError`` is checked first and by type, not by
|
|
224
|
+
message text. It carries ``status_code=None`` and would otherwise land in
|
|
225
|
+
the generic ``upstream-error`` bucket, hiding the one failure mode this
|
|
226
|
+
reason exists to name. The import is function-local because
|
|
227
|
+
``adapters.base`` is a TYPE_CHECKING-only import at module scope here
|
|
228
|
+
(the runtime import order would otherwise cycle through the registry).
|
|
229
|
+
"""
|
|
230
|
+
from coderouter.adapters.base import StreamTruncatedError
|
|
231
|
+
|
|
232
|
+
if isinstance(exc, StreamTruncatedError):
|
|
233
|
+
return REASON_STREAM_TRUNCATED
|
|
234
|
+
status = exc.status_code
|
|
235
|
+
if status is None:
|
|
236
|
+
message = str(exc).lower()
|
|
237
|
+
if "timeout" in message or "timed out" in message:
|
|
238
|
+
return REASON_TIMEOUT
|
|
239
|
+
if "transport error" in message or "connect" in message:
|
|
240
|
+
return REASON_CONNECTION
|
|
241
|
+
return REASON_UPSTREAM_ERROR
|
|
242
|
+
if status == 408:
|
|
243
|
+
return REASON_TIMEOUT
|
|
244
|
+
if status == 429:
|
|
245
|
+
return REASON_RATE_LIMIT
|
|
246
|
+
if status in {401, 403}:
|
|
247
|
+
return REASON_AUTH
|
|
248
|
+
if status >= 500:
|
|
249
|
+
return REASON_UPSTREAM_5XX
|
|
250
|
+
if status >= 400:
|
|
251
|
+
return REASON_UPSTREAM_4XX
|
|
252
|
+
return REASON_UPSTREAM_ERROR
|
|
253
|
+
|
|
254
|
+
|
|
255
|
+
@dataclass(frozen=True)
|
|
256
|
+
class FallbackHop:
|
|
257
|
+
"""One provider→provider transition and the reason behind it.
|
|
258
|
+
|
|
259
|
+
``to_provider`` is ``None`` until the *next* provider is actually
|
|
260
|
+
attempted (see :meth:`FallbackTrace.record_attempt`); it stays ``None``
|
|
261
|
+
forever when the chain was exhausted without a successor.
|
|
262
|
+
"""
|
|
263
|
+
|
|
264
|
+
from_provider: str
|
|
265
|
+
reason: str
|
|
266
|
+
to_provider: str | None = None
|
|
267
|
+
detail: str | None = None
|
|
268
|
+
stream: bool = False
|
|
269
|
+
pre_attempt: bool = False
|
|
270
|
+
|
|
271
|
+
def as_dict(self) -> dict[str, Any]:
|
|
272
|
+
"""Serializable view, used by the SSE metadata event."""
|
|
273
|
+
return {
|
|
274
|
+
"from": self.from_provider,
|
|
275
|
+
"to": self.to_provider,
|
|
276
|
+
"reason": self.reason,
|
|
277
|
+
"detail": self.detail,
|
|
278
|
+
"stream": self.stream,
|
|
279
|
+
"pre_attempt": self.pre_attempt,
|
|
280
|
+
}
|
|
281
|
+
|
|
282
|
+
|
|
283
|
+
@dataclass
|
|
284
|
+
class FallbackTrace:
|
|
285
|
+
"""Ordered record of every fallback that happened while serving one request.
|
|
286
|
+
|
|
287
|
+
Mutated in place by the engine. The ingress only ever reads it, so the
|
|
288
|
+
object can be shared safely between the request-handler context and the
|
|
289
|
+
SSE-iterator context (which inherits a copy of the same ContextVar
|
|
290
|
+
mapping, and therefore the same object).
|
|
291
|
+
"""
|
|
292
|
+
|
|
293
|
+
hops: list[FallbackHop] = field(default_factory=list)
|
|
294
|
+
attempts: list[str] = field(default_factory=list)
|
|
295
|
+
profile: str | None = None
|
|
296
|
+
#: Set once the engine has emitted the ``fallback-occurred`` log lines
|
|
297
|
+
#: for this request, so a dispatch path with several terminal returns
|
|
298
|
+
#: cannot log the same hop trail twice.
|
|
299
|
+
logged: bool = False
|
|
300
|
+
|
|
301
|
+
# -- recording ---------------------------------------------------------
|
|
302
|
+
|
|
303
|
+
def record_skip(
|
|
304
|
+
self, provider: str, reason: str, *, detail: str | None = None
|
|
305
|
+
) -> None:
|
|
306
|
+
"""Record a provider filtered out of the chain before it was called."""
|
|
307
|
+
self.hops.append(
|
|
308
|
+
FallbackHop(
|
|
309
|
+
from_provider=provider,
|
|
310
|
+
reason=reason,
|
|
311
|
+
detail=detail,
|
|
312
|
+
pre_attempt=True,
|
|
313
|
+
)
|
|
314
|
+
)
|
|
315
|
+
|
|
316
|
+
def record_failure(
|
|
317
|
+
self,
|
|
318
|
+
provider: str,
|
|
319
|
+
reason: str,
|
|
320
|
+
*,
|
|
321
|
+
detail: str | None = None,
|
|
322
|
+
stream: bool = False,
|
|
323
|
+
) -> None:
|
|
324
|
+
"""Record a provider that was called and did not deliver."""
|
|
325
|
+
self.hops.append(
|
|
326
|
+
FallbackHop(
|
|
327
|
+
from_provider=provider,
|
|
328
|
+
reason=reason,
|
|
329
|
+
detail=detail,
|
|
330
|
+
stream=stream,
|
|
331
|
+
pre_attempt=False,
|
|
332
|
+
)
|
|
333
|
+
)
|
|
334
|
+
|
|
335
|
+
def record_attempt(self, provider: str) -> None:
|
|
336
|
+
"""Record that ``provider`` is being tried, resolving pending hops.
|
|
337
|
+
|
|
338
|
+
Every hop still waiting for a successor (``to_provider is None``)
|
|
339
|
+
gets ``provider`` filled in. Consecutive skips therefore all point
|
|
340
|
+
at the same provider that finally ran, which is exactly the
|
|
341
|
+
narrative an operator wants: "budget blocked local and health
|
|
342
|
+
blocked ollama, so openrouter served it".
|
|
343
|
+
"""
|
|
344
|
+
self.attempts.append(provider)
|
|
345
|
+
self.resolve_pending(provider)
|
|
346
|
+
|
|
347
|
+
def resolve_pending(self, provider: str) -> None:
|
|
348
|
+
"""Point every still-unresolved hop at ``provider``.
|
|
349
|
+
|
|
350
|
+
Split out of :meth:`record_attempt` because the *streaming* ingress
|
|
351
|
+
needs the answer earlier than the first attempt: an SSE response
|
|
352
|
+
commits its HTTP headers before the engine's generator runs a
|
|
353
|
+
single line, so ``X-CodeRouter-Fallback-To`` would always be
|
|
354
|
+
missing if only ``record_attempt`` could fill it. The engine calls
|
|
355
|
+
this from ``apply_context_budget`` — the one place where the chain
|
|
356
|
+
is fully ordered (adaptive reorder, drift demotion and capability
|
|
357
|
+
bucketing have all been applied) and its head is therefore
|
|
358
|
+
genuinely the provider that will be tried first.
|
|
359
|
+
|
|
360
|
+
Idempotent and monotonic: only hops holding ``None`` are touched,
|
|
361
|
+
so a later ``record_attempt`` can never rewrite an answer this
|
|
362
|
+
already gave, and the two mechanisms compose instead of fighting.
|
|
363
|
+
"""
|
|
364
|
+
for index, hop in enumerate(self.hops):
|
|
365
|
+
if hop.to_provider is None:
|
|
366
|
+
self.hops[index] = FallbackHop(
|
|
367
|
+
from_provider=hop.from_provider,
|
|
368
|
+
reason=hop.reason,
|
|
369
|
+
to_provider=provider,
|
|
370
|
+
detail=hop.detail,
|
|
371
|
+
stream=hop.stream,
|
|
372
|
+
pre_attempt=hop.pre_attempt,
|
|
373
|
+
)
|
|
374
|
+
|
|
375
|
+
# -- reading -----------------------------------------------------------
|
|
376
|
+
|
|
377
|
+
@property
|
|
378
|
+
def occurred(self) -> bool:
|
|
379
|
+
"""True iff at least one fallback hop was recorded."""
|
|
380
|
+
return bool(self.hops)
|
|
381
|
+
|
|
382
|
+
@property
|
|
383
|
+
def chain(self) -> list[str]:
|
|
384
|
+
"""Providers in try-order, including the one that finally served."""
|
|
385
|
+
if not self.hops:
|
|
386
|
+
return list(self.attempts)
|
|
387
|
+
names = [hop.from_provider for hop in self.hops]
|
|
388
|
+
final = self.hops[-1].to_provider
|
|
389
|
+
if final is not None:
|
|
390
|
+
names.append(final)
|
|
391
|
+
return names
|
|
392
|
+
|
|
393
|
+
@property
|
|
394
|
+
def reasons(self) -> list[str]:
|
|
395
|
+
"""Departure reason for each hop, in order."""
|
|
396
|
+
return [hop.reason for hop in self.hops]
|
|
397
|
+
|
|
398
|
+
@property
|
|
399
|
+
def from_provider(self) -> str | None:
|
|
400
|
+
"""The first provider that was abandoned."""
|
|
401
|
+
return self.hops[0].from_provider if self.hops else None
|
|
402
|
+
|
|
403
|
+
@property
|
|
404
|
+
def to_provider(self) -> str | None:
|
|
405
|
+
"""The provider that ultimately served, or ``None`` if none did."""
|
|
406
|
+
return self.hops[-1].to_provider if self.hops else None
|
|
407
|
+
|
|
408
|
+
def header_values(self) -> dict[str, str]:
|
|
409
|
+
"""Build the ``X-CodeRouter-Fallback-*`` response headers.
|
|
410
|
+
|
|
411
|
+
Returns an empty dict when no fallback occurred, so the caller can
|
|
412
|
+
merge unconditionally and the zero-fallback response keeps exactly
|
|
413
|
+
the headers it had before v2.15.0.
|
|
414
|
+
"""
|
|
415
|
+
if not self.hops:
|
|
416
|
+
return {}
|
|
417
|
+
headers: dict[str, str] = {
|
|
418
|
+
HEADER_FALLBACK_FROM: _sanitize(self.hops[0].from_provider),
|
|
419
|
+
HEADER_FALLBACK_REASON: ",".join(_sanitize(r) for r in self.reasons),
|
|
420
|
+
HEADER_FALLBACK_CHAIN: ">".join(_sanitize(p) for p in self.chain),
|
|
421
|
+
}
|
|
422
|
+
final = self.hops[-1].to_provider
|
|
423
|
+
if final is not None:
|
|
424
|
+
headers[HEADER_FALLBACK_TO] = _sanitize(final)
|
|
425
|
+
return headers
|
|
426
|
+
|
|
427
|
+
def as_event_payload(self) -> dict[str, Any]:
|
|
428
|
+
"""Build the ``coderouter_fallback`` SSE metadata event body."""
|
|
429
|
+
return {
|
|
430
|
+
"type": SSE_FALLBACK_EVENT,
|
|
431
|
+
"from": self.from_provider,
|
|
432
|
+
"to": self.to_provider,
|
|
433
|
+
"reason": self.reasons,
|
|
434
|
+
"chain": self.chain,
|
|
435
|
+
"hops": [hop.as_dict() for hop in self.hops],
|
|
436
|
+
}
|
|
437
|
+
|
|
438
|
+
|
|
439
|
+
# ---------------------------------------------------------------------------
|
|
440
|
+
# Request-scoped storage
|
|
441
|
+
# ---------------------------------------------------------------------------
|
|
442
|
+
|
|
443
|
+
_fallback_trace_ctx: ContextVar[FallbackTrace | None] = ContextVar(
|
|
444
|
+
"coderouter_fallback_trace", default=None
|
|
445
|
+
)
|
|
446
|
+
|
|
447
|
+
|
|
448
|
+
def begin_fallback_trace(
|
|
449
|
+
*, keep_existing: bool = False, profile: str | None = None
|
|
450
|
+
) -> FallbackTrace:
|
|
451
|
+
"""Install (or reuse) the trace for the request being dispatched.
|
|
452
|
+
|
|
453
|
+
``keep_existing=True`` is used by the engine entry points when the M11
|
|
454
|
+
prepared dispatch was consumed: that proves the ingress already called
|
|
455
|
+
``apply_context_budget`` for *this* request, so the trace it started
|
|
456
|
+
(holding the chain-resolve skips) must be kept rather than replaced.
|
|
457
|
+
|
|
458
|
+
In every other case a fresh trace is installed, which is what makes
|
|
459
|
+
successive direct engine calls in one context (unit tests, scripts)
|
|
460
|
+
independent instead of accumulating hops forever.
|
|
461
|
+
"""
|
|
462
|
+
existing = _fallback_trace_ctx.get()
|
|
463
|
+
if keep_existing and existing is not None:
|
|
464
|
+
if profile is not None:
|
|
465
|
+
existing.profile = profile
|
|
466
|
+
return existing
|
|
467
|
+
trace = FallbackTrace(profile=profile)
|
|
468
|
+
_fallback_trace_ctx.set(trace)
|
|
469
|
+
return trace
|
|
470
|
+
|
|
471
|
+
|
|
472
|
+
def ensure_fallback_trace(*, profile: str | None = None) -> FallbackTrace:
|
|
473
|
+
"""Return the current trace, creating one only if none exists.
|
|
474
|
+
|
|
475
|
+
Used by chain resolution, which runs both inside dispatch (a trace
|
|
476
|
+
already exists) and from the ingress's ``apply_context_budget``
|
|
477
|
+
pre-pass (it does not yet).
|
|
478
|
+
"""
|
|
479
|
+
existing = _fallback_trace_ctx.get()
|
|
480
|
+
if existing is not None:
|
|
481
|
+
if profile is not None and existing.profile is None:
|
|
482
|
+
existing.profile = profile
|
|
483
|
+
return existing
|
|
484
|
+
trace = FallbackTrace(profile=profile)
|
|
485
|
+
_fallback_trace_ctx.set(trace)
|
|
486
|
+
return trace
|
|
487
|
+
|
|
488
|
+
|
|
489
|
+
def current_fallback_trace() -> FallbackTrace | None:
|
|
490
|
+
"""Return the trace for the current request, or ``None``.
|
|
491
|
+
|
|
492
|
+
Read-only accessor for the ingress. ``None`` means "nothing recorded"
|
|
493
|
+
— either no fallback machinery ran (a stubbed engine in a test) or the
|
|
494
|
+
engine never started a trace. Callers must treat it as "no fallback".
|
|
495
|
+
"""
|
|
496
|
+
return _fallback_trace_ctx.get()
|
|
497
|
+
|
|
498
|
+
|
|
499
|
+
def reset_fallback_trace() -> None:
|
|
500
|
+
"""Clear the current trace. Test helper; not used in the request path."""
|
|
501
|
+
_fallback_trace_ctx.set(None)
|
coderouter/state/audit_log.py
CHANGED
|
@@ -78,6 +78,10 @@ _AUDIT_EVENTS: frozenset[str] = frozenset(
|
|
|
78
78
|
"chain-paid-gate-blocked",
|
|
79
79
|
"chain-memory-pressure-blocked",
|
|
80
80
|
"chain-uniform-auth-failure",
|
|
81
|
+
# Fallback reason trail (v2.15.0) — one line per chain transition.
|
|
82
|
+
# Low volume by construction: nothing is emitted when the first
|
|
83
|
+
# provider serves the request.
|
|
84
|
+
"fallback-occurred",
|
|
81
85
|
# Memory pressure
|
|
82
86
|
"memory-pressure-detected",
|
|
83
87
|
# Drift (v2.0-G)
|
coderouter/vscode_init.py
CHANGED
|
@@ -24,9 +24,16 @@ The design contract:
|
|
|
24
24
|
until a human debugged it.
|
|
25
25
|
* Nothing an operator wrote is destroyed without a copy: before any
|
|
26
26
|
existing file is rewritten, its previous bytes (and mode) land in
|
|
27
|
-
``<name>.bak
|
|
28
|
-
|
|
29
|
-
|
|
27
|
+
``<name>.bak``, which ``coderouter rollback`` can swap back.
|
|
28
|
+
* Since v2.14.0, ``.envrc`` is edited through a marker-delimited block
|
|
29
|
+
(``# BEGIN coderouter-managed`` … ``# END coderouter-managed``): only
|
|
30
|
+
that block is rewritten and every other line survives byte-for-byte.
|
|
31
|
+
A re-run over an already-fenced file therefore needs no ``--force``;
|
|
32
|
+
``--force`` now means "adopt a file I did not write". Before v2.14.0
|
|
33
|
+
this was a whole-file replace, which is how the lost
|
|
34
|
+
``source_env_if_exists .envrc.local`` line (H-11) happened — see the
|
|
35
|
+
"marker-delimited managed block in .envrc" section below for the full
|
|
36
|
+
rationale and the conflict rules.
|
|
30
37
|
* A freshly generated ``.envrc`` is created 0600 — it carries
|
|
31
38
|
``ANTHROPIC_AUTH_TOKEN`` and sits next to the operator's real
|
|
32
39
|
secrets. An existing file's mode is preserved across rewrites
|
|
@@ -1,6 +1,6 @@
|
|
|
1
|
-
Metadata-Version: 2.
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
2
|
Name: coderouter-cli
|
|
3
|
-
Version: 2.
|
|
3
|
+
Version: 2.15.1
|
|
4
4
|
Summary: Local-first, free-first, fallback-built-in LLM router. Claude Code / OpenAI compatible.
|
|
5
5
|
Project-URL: Homepage, https://github.com/zephel01/CodeRouter
|
|
6
6
|
Project-URL: Repository, https://github.com/zephel01/CodeRouter
|
|
@@ -66,7 +66,7 @@ Description-Content-Type: text/markdown
|
|
|
66
66
|
## 何ができるか — 30 秒で
|
|
67
67
|
|
|
68
68
|
```
|
|
69
|
-
あなたのエージェント (Claude Code /
|
|
69
|
+
あなたのエージェント (Claude Code / agy ほか ※)
|
|
70
70
|
│
|
|
71
71
|
▼
|
|
72
72
|
┌─ CodeRouter ─┐
|
|
@@ -76,6 +76,8 @@ Description-Content-Type: text/markdown
|
|
|
76
76
|
└──────────────┘
|
|
77
77
|
```
|
|
78
78
|
|
|
79
|
+
※ 現行の codex CLI は Responses API(`/v1/responses`)専用になったため、CodeRouter 経由ではまだ使えません(対応予定)。直結の代替手順 → [quickstart Pattern B](./docs/start/quickstart.md#pattern-b-codex-cli-で使う)
|
|
80
|
+
|
|
79
81
|
**やってくれること:**
|
|
80
82
|
|
|
81
83
|
- ローカルモデルが壊した tool calling を Claude Code に届く前に修復する
|
|
@@ -118,6 +120,8 @@ uvx --from coderouter-cli coderouter serve --port 8088
|
|
|
118
120
|
|
|
119
121
|
恒久インストールしたい場合: `uv tool install coderouter-cli`
|
|
120
122
|
|
|
123
|
+
> **パッケージ名に注意**: PyPI のパッケージ名は **`coderouter-cli`** です。PyPI の `coderouter`(名前だけ同じ)は別の製品なので、`pip install coderouter` と打たないでください。
|
|
124
|
+
|
|
121
125
|
---
|
|
122
126
|
|
|
123
127
|
## Claude Code で使う
|
|
@@ -143,7 +147,7 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
|
|
|
143
147
|
| Claude Code + ローカル Ollama で tool calling が壊れる | **必須** — tool 修復 (+ 必要なら wire 変換) |
|
|
144
148
|
| Claude Code + ローカルで長時間回すと止まる | **必須級** — 6 系統ガード + self-healing |
|
|
145
149
|
| Ollama v0.14+ / LM Studio にネイティブ直結で動いてる | **便利** — 直結に無い fallback / ガード / 診断を追加 (passthrough で翻訳ゼロ) |
|
|
146
|
-
| codex / agy + Ollama 直繋ぎで動いてる | オプション — フォールバックが欲しいなら |
|
|
150
|
+
| codex / agy + Ollama 直繋ぎで動いてる | オプション — フォールバックが欲しいなら(codex は `/v1/responses` 対応後) |
|
|
147
151
|
| Claude API を直接叩いてて問題ない | 不要 |
|
|
148
152
|
|
|
149
153
|
詳細は → [要否判定ガイド](./docs/start/when-do-i-need-coderouter.md)
|
|
@@ -156,7 +160,7 @@ ANTHROPIC_BASE_URL=http://localhost:8088 ANTHROPIC_AUTH_TOKEN=dummy claude
|
|
|
156
160
|
|
|
157
161
|
| 機能 | 何をしてくれるか |
|
|
158
162
|
|---|---|
|
|
159
|
-
| **Tool-call 修復** | ローカルモデルがテキストで吐いた JSON を正しい tool_use
|
|
163
|
+
| **Tool-call 修復** | ローカルモデルがテキストで吐いた JSON を正しい tool_use ブロックに復元。実測は [修復ベンチ](./benchmarks/tool-repair/README.md)(壊れた tool call を出す 4 モデルで成功率 0% → 80〜100%、回帰コーパスで誤修復 0 件。CI でゲート) |
|
|
160
164
|
| **3 層フォールバック** | ローカル → 無料クラウド → 有料の順に自動切替 |
|
|
161
165
|
| **出力フィルタ** | `<think>` タグ漏れ、stop marker 漏れ、byte-fallback (`<0xNN>`) を自動除去/修復 |
|
|
162
166
|
| **Wire 翻訳** | Anthropic 形式 ↔ OpenAI 形式を自動変換 (ネイティブ `/v1/messages` 対応 backend は passthrough で翻訳ゼロ) |
|
|
@@ -333,6 +337,14 @@ CodeRouter は backend ルーター層として独立して動きます。`OPENA
|
|
|
333
337
|
|
|
334
338
|
シークレットは環境変数に置きます。[`docs/security.md`](./docs/guides/security.md) に完全な方針と報告手順があります。
|
|
335
339
|
|
|
340
|
+
リリースは多層で守っています。どれか 1 つを根拠に「安全」とは言いません(正規のパイプラインでビルドされた汚染版が出た例があります)。
|
|
341
|
+
|
|
342
|
+
- PyPI へは Trusted Publishing(OIDC)で公開し、長期の API トークンを置かない
|
|
343
|
+
- PyPI 側の公開許可は `release.yml` の `pypi` 環境に限定している
|
|
344
|
+
- タグは事前検査スクリプトを通してから打つ(version・CHANGELOG・main の一致を機械的に確認)
|
|
345
|
+
- 実行時の依存は 5 個に絞っている
|
|
346
|
+
- 各リリースに SBOM(CycloneDX)を添付する
|
|
347
|
+
|
|
336
348
|
## License
|
|
337
349
|
|
|
338
350
|
MIT
|