@oneciel-ai/claude-any 0.1.35 → 0.1.37

package/README.md

@@ -48,7 +48,7 @@ arguments through unchanged.
 
 Credits: One Ciel LLC
 
-Current version: `0.1.35`
+Current version: `0.1.37`
 
 ## Why This Exists
 
@@ -381,6 +381,22 @@ steps under that larger model's supervision.
 
 ## Changelog
 
+### 0.1.37
+
+- **Pseudo tool-call recovery**: the NVIDIA/OpenAI-compatible stream path now
+  suppresses `<|tool_calls_section_begin|>...` pseudo tool-call text and
+  converts it back into Claude `tool_use` blocks when possible.
+- **Streaming defaults**: provider streaming defaults to on; NVIDIA hosted
+  remains forced to the streaming upstream path for stability.
+
+### 0.1.36
+
+- **NVIDIA upstream streaming**: NVIDIA hosted router calls now use upstream
+  `stream=true`, so long responses can flow as chunks instead of waiting for a
+  full non-streaming completion.
+- **Stream retry diagnostics**: streamed NVIDIA calls keep the same retry and
+  request-size activity status used by the statusline.
+
 ### 0.1.35
 
 - **NVIDIA router context guard**: NVIDIA hosted now defaults to a 32K router

package/claude_any.py

@@ -85,7 +85,7 @@ PROVIDER_LABELS = {
     "self-hosted-nim": "Self Hosted NIM",
 }
 APP_NAME = "Claude Any"
-VERSION = "0.1.35"
+VERSION = "0.1.37"
 CREDITS = "Credits: One Ciel LLC"
 
 LOG_LEVELS = {"SILENT": 0, "ERROR": 1, "WARN": 2, "INFO": 3, "DEBUG": 4, "TRACE": 5}
@@ -790,6 +790,13 @@ def apply_config_migrations(cfg: dict[str, Any]) -> None:
         if isinstance(pcfg, dict) and not positive_int(pcfg.get("context_window")):
             pcfg["context_window"] = 32768
         migrations[marker] = True
+
+    marker = "stream_enabled_default_true_20260513"
+    if not migrations.get(marker):
+        for pcfg in (cfg.get("providers") or {}).values():
+            if isinstance(pcfg, dict) and "stream_enabled" not in pcfg:
+                pcfg["stream_enabled"] = True
+        migrations[marker] = True
 
 
 _config_cache: dict[str, Any] | None = None
@@ -3988,7 +3995,7 @@ def maybe_handle_advisor_request(handler: BaseHTTPRequestHandler, provider: str,
     return True
 
 
-def normalize_tool_arguments(tool_name: str, args: Any) -> dict[str, Any]:
+def normalize_tool_arguments(tool_name: str, args: Any) -> dict[str, Any]:
     if isinstance(args, dict):
         return args
     if isinstance(args, str):
@@ -4003,17 +4010,86 @@ def normalize_tool_arguments(tool_name: str, args: Any) -> dict[str, Any]:
             pass
         if tool_name == "Bash":
             return {"command": text}
-    return {}
-
-
-def ollama_chat_to_anthropic(data: dict[str, Any], model: str, source_body: dict[str, Any] | None = None) -> dict[str, Any]:
-    message = data.get("message") if isinstance(data.get("message"), dict) else {}
-    content: list[dict[str, Any]] = []
-    text = message.get("content") or ""
-    if text:
-        content.append({"type": "text", "text": text})
-    tool_id_prefix = f"toolu_ollama_{int(time.time() * 1000)}_{os.getpid()}"
-    for i, call in enumerate(message.get("tool_calls") or []):
+    return {}
+
+
+PSEUDO_TOOL_START = "<|tool_calls_section_begin|>"
+PSEUDO_TOOL_END = "<|tool_calls_section_end|>"
+PSEUDO_CALL_BEGIN = "<|tool_call_begin|>"
+PSEUDO_ARG_BEGIN = "<|tool_call_argument_begin|>"
+PSEUDO_CALL_END = "<|tool_call_end|>"
+
+
+def infer_tool_name_from_args(args: dict[str, Any]) -> str:
+    keys = set(args)
+    if "command" in keys:
+        return "Bash"
+    if {"file_path", "content"}.issubset(keys):
+        return "Write"
+    if {"file_path", "old_string", "new_string"}.issubset(keys):
+        return "Edit"
+    if "file_path" in keys:
+        return "Read"
+    if "taskId" in keys and "status" in keys:
+        return "TaskUpdate"
+    return "TaskList" if not args else "Write"
+
+
+def parse_pseudo_tool_calls(text: str) -> tuple[str, list[dict[str, Any]]]:
+    if PSEUDO_TOOL_START not in text:
+        return text, []
+    visible_parts: list[str] = []
+    calls: list[dict[str, Any]] = []
+    pos = 0
+    while True:
+        start = text.find(PSEUDO_TOOL_START, pos)
+        if start < 0:
+            visible_parts.append(text[pos:])
+            break
+        visible_parts.append(text[pos:start])
+        end = text.find(PSEUDO_TOOL_END, start)
+        if end < 0:
+            section = text[start + len(PSEUDO_TOOL_START):]
+            pos = len(text)
+        else:
+            section = text[start + len(PSEUDO_TOOL_START):end]
+            pos = end + len(PSEUDO_TOOL_END)
+        for match in re.finditer(
+            re.escape(PSEUDO_CALL_BEGIN) + r"(.*?)" + re.escape(PSEUDO_ARG_BEGIN) + r"(.*?)" + re.escape(PSEUDO_CALL_END),
+            section,
+            flags=re.DOTALL,
+        ):
+            raw_header = match.group(1).strip()
+            raw_args = match.group(2).strip()
+            try:
+                args = json.loads(raw_args)
+            except Exception:
+                continue
+            if not isinstance(args, dict):
+                continue
+            name = ""
+            for part in re.split(r"[\s:|,]+", raw_header):
+                candidate = _fuzzy_match_tool_name(part)
+                if candidate:
+                    name = candidate
+                    break
+            if not name:
+                name = infer_tool_name_from_args(args)
+            calls.append({"function": {"name": name, "arguments": args}, "id": raw_header})
+        if end < 0:
+            break
+    return "".join(visible_parts), calls
+
+
+def ollama_chat_to_anthropic(data: dict[str, Any], model: str, source_body: dict[str, Any] | None = None) -> dict[str, Any]:
+    message = data.get("message") if isinstance(data.get("message"), dict) else {}
+    content: list[dict[str, Any]] = []
+    text = message.get("content") or ""
+    text, pseudo_tool_calls = parse_pseudo_tool_calls(text)
+    if text:
+        content.append({"type": "text", "text": text})
+    tool_id_prefix = f"toolu_ollama_{int(time.time() * 1000)}_{os.getpid()}"
+    for i, call in enumerate(list(message.get("tool_calls") or []) + pseudo_tool_calls):
         fn = call.get("function") if isinstance(call, dict) else {}
         if not isinstance(fn, dict) or not fn.get("name"):
             continue
@@ -4579,7 +4655,7 @@ def forward_ollama_api_chat(handler: BaseHTTPRequestHandler, provider: str, pcfg
     write_json(handler, message)
 
 
-def openai_chat_to_anthropic(data: dict[str, Any], model: str, source_body: dict[str, Any] | None = None) -> dict[str, Any]:
+def openai_chat_to_anthropic(data: dict[str, Any], model: str, source_body: dict[str, Any] | None = None) -> dict[str, Any]:
     choice = {}
     choices = data.get("choices")
     if isinstance(choices, list) and choices:
@@ -4592,10 +4668,228 @@ def openai_chat_to_anthropic(data: dict[str, Any], model: str, source_body: dict
         },
         "done_reason": "length" if choice.get("finish_reason") == "length" else "stop",
     }
-    return ollama_chat_to_anthropic(wrapped, model, source_body=source_body)
-
-
-def upstream_http_error_message(exc: urllib.error.HTTPError, raw: str | None = None) -> str:
+    return ollama_chat_to_anthropic(wrapped, model, source_body=source_body)
+
+
+def stream_openai_chat_to_anthropic_sse(
+    handler: BaseHTTPRequestHandler,
+    resp: Any,
+    model: str,
+    provider: str,
+    source_body: dict[str, Any] | None = None,
+    start_index: int = 0,
+    word_chunking: bool = False,
+) -> None:
+    next_content_index = start_index
+    text_started = False
+    text_suppressed_for_plan = False
+    text_index: int | None = None
+    text_so_far = ""
+    pseudo_text = ""
+    pseudo_mode = False
+    text_buffer = ""
+    tool_fragments: dict[int, dict[str, Any]] = {}
+    output_tokens = 0
+    finish_reason = "stop"
+
+    def emit(event_name: str, payload: dict[str, Any]) -> None:
+        handler.wfile.write(f"event: {event_name}\ndata: {json.dumps(payload, ensure_ascii=False)}\n\n".encode())
+        handler.wfile.flush()
+
+    def ensure_text_started() -> int:
+        nonlocal text_started, text_index, next_content_index
+        if text_started and text_index is not None:
+            return text_index
+        text_started = True
+        text_index = next_content_index
+        next_content_index += 1
+        emit(
+            "content_block_start",
+            {"type": "content_block_start", "index": text_index, "content_block": {"type": "text", "text": ""}},
+        )
+        return text_index
+
+    def emit_text_delta(text: str) -> None:
+        if not text:
+            return
+        idx = ensure_text_started()
+        emit(
+            "content_block_delta",
+            {"type": "content_block_delta", "index": idx, "delta": {"type": "text_delta", "text": text}},
+        )
+
+    try:
+        for raw_line in resp:
+            line = raw_line.decode("utf-8", errors="ignore").strip()
+            if not line or line.startswith(":"):
+                continue
+            if line.startswith("data:"):
+                line = line[5:].strip()
+            if not line or line == "[DONE]":
+                break
+            try:
+                event = json.loads(line)
+            except Exception:
+                continue
+            if not isinstance(event, dict):
+                continue
+            usage = event.get("usage")
+            if isinstance(usage, dict):
+                output_tokens = max(output_tokens, positive_int(usage.get("completion_tokens")) or 0)
+            choices = event.get("choices")
+            if not isinstance(choices, list) or not choices:
+                continue
+            choice = choices[0] if isinstance(choices[0], dict) else {}
+            if choice.get("finish_reason"):
+                finish_reason = str(choice.get("finish_reason"))
+            delta = choice.get("delta") if isinstance(choice.get("delta"), dict) else {}
+            text_chunk = delta.get("content") or ""
+            if text_chunk:
+                if pseudo_mode or PSEUDO_TOOL_START in text_chunk:
+                    before, sep, after = text_chunk.partition(PSEUDO_TOOL_START)
+                    if before and not pseudo_mode:
+                        text_so_far += before
+                        if word_chunking:
+                            text_buffer += before
+                            to_flush, text_buffer = _split_word_buffer(text_buffer, force=False)
+                            emit_text_delta(to_flush)
+                        else:
+                            emit_text_delta(before)
+                    pseudo_mode = True
+                    pseudo_text += (sep + after) if sep else text_chunk
+                    if PSEUDO_TOOL_END in pseudo_text:
+                        pseudo_mode = False
+                    continue
+                if source_body is not None and not text_started and not tool_fragments and should_auto_enter_plan_mode(source_body, text_so_far + text_chunk, []):
+                    text_so_far += text_chunk
+                    text_suppressed_for_plan = True
+                    continue
+                text_so_far += text_chunk
+                if word_chunking:
+                    text_buffer += text_chunk
+                    to_flush, text_buffer = _split_word_buffer(text_buffer, force=False)
+                    emit_text_delta(to_flush)
+                else:
+                    emit_text_delta(text_chunk)
+            for call in delta.get("tool_calls") or []:
+                if not isinstance(call, dict):
+                    continue
+                try:
+                    call_index = int(call.get("index"))
+                except Exception:
+                    call_index = len(tool_fragments)
+                slot = tool_fragments.setdefault(call_index, {"id": "", "name": "", "arguments": ""})
+                if call.get("id"):
+                    slot["id"] = str(call.get("id"))
+                fn = call.get("function") if isinstance(call.get("function"), dict) else {}
+                if fn.get("name"):
+                    slot["name"] += str(fn.get("name"))
+                if fn.get("arguments"):
+                    slot["arguments"] += str(fn.get("arguments"))
+        if word_chunking and text_buffer:
+            to_flush, text_buffer = _split_word_buffer(text_buffer, force=True)
+            emit_text_delta(to_flush)
+
+        tool_calls: list[dict[str, Any]] = []
+        _, pseudo_tool_calls = parse_pseudo_tool_calls(pseudo_text)
+        for i, pseudo in enumerate(pseudo_tool_calls):
+            fn = pseudo.get("function") if isinstance(pseudo, dict) else {}
+            if isinstance(fn, dict):
+                tool_fragments.setdefault(100000 + i, {
+                    "id": str(pseudo.get("id") or ""),
+                    "name": str(fn.get("name") or ""),
+                    "arguments": json.dumps(fn.get("arguments") or {}, ensure_ascii=False),
+                })
+        for _, fragment in sorted(tool_fragments.items()):
+            raw_name = str(fragment.get("name") or "")
+            if not raw_name:
+                continue
+            matched_name = _fuzzy_match_tool_name(raw_name) or raw_name
+            normalized_args = normalize_tool_arguments(matched_name, fragment.get("arguments") or {})
+            fixed_input = _validate_and_fix_tool_input(matched_name, normalized_args)
+            if source_body is not None:
+                matched_name, fixed_input = plan_mode_tool_name_for_emit(source_body, matched_name, fixed_input)
+                if matched_name is None:
+                    continue
+            tool_calls.append({"function": {"name": matched_name, "arguments": fixed_input}})
+            tool_index = next_content_index
+            next_content_index += 1
+            tool_id = str(fragment.get("id") or f"toolu_openai_{int(time.time() * 1000)}_{tool_index}")
+            append_tool_call_log(
+                "openai_stream_tool_call",
+                {
+                    "model": model,
+                    "raw_name": raw_name,
+                    "matched_name": matched_name,
+                    "raw_arguments": fragment.get("arguments"),
+                    "emitted_input": fixed_input,
+                    "sse_index": tool_index,
+                },
+            )
+            emit(
+                "content_block_start",
+                {
+                    "type": "content_block_start",
+                    "index": tool_index,
+                    "content_block": {"type": "tool_use", "id": tool_id, "name": matched_name, "input": {}},
+                },
+            )
+            emit(
+                "content_block_delta",
+                {
+                    "type": "content_block_delta",
+                    "index": tool_index,
+                    "delta": {"type": "input_json_delta", "partial_json": json.dumps(fixed_input, ensure_ascii=False)},
+                },
+            )
+            emit("content_block_stop", {"type": "content_block_stop", "index": tool_index})
+
+        if source_body is not None and should_auto_enter_plan_mode(source_body, text_so_far, tool_calls):
+            router_log("WARN", "auto-synthesized EnterPlanMode from short/empty upstream OpenAI stream")
+            tool_index = next_content_index
+            next_content_index += 1
+            tool_calls.append({"function": {"name": "EnterPlanMode", "arguments": {}}})
+            emit(
+                "content_block_start",
+                {
+                    "type": "content_block_start",
+                    "index": tool_index,
+                    "content_block": {"type": "tool_use", "id": f"toolu_openai_plan_{int(time.time() * 1000)}", "name": "EnterPlanMode", "input": {}},
+                },
+            )
+            emit("content_block_delta", {"type": "content_block_delta", "index": tool_index, "delta": {"type": "input_json_delta", "partial_json": "{}"}})
+            emit("content_block_stop", {"type": "content_block_stop", "index": tool_index})
+        elif text_suppressed_for_plan and text_so_far:
+            emit_text_delta(text_so_far)
+
+        if source_body is not None and should_keep_work_alive_with_tasklist(source_body, text_so_far, tool_calls):
+            router_log("WARN", "auto-synthesized TaskList to keep work moving after OpenAI stream")
+            tool_index = next_content_index
+            next_content_index += 1
+            tool_calls.append({"function": {"name": "TaskList", "arguments": {}}})
+            emit(
+                "content_block_start",
+                {
+                    "type": "content_block_start",
+                    "index": tool_index,
+                    "content_block": {"type": "tool_use", "id": f"toolu_openai_keepalive_{int(time.time() * 1000)}", "name": "TaskList", "input": {}},
+                },
+            )
+            emit("content_block_delta", {"type": "content_block_delta", "index": tool_index, "delta": {"type": "input_json_delta", "partial_json": "{}"}})
+            emit("content_block_stop", {"type": "content_block_stop", "index": tool_index})
+
+        if text_started and text_index is not None:
+            emit("content_block_stop", {"type": "content_block_stop", "index": text_index})
+        stop_reason = "tool_use" if tool_calls else ("max_tokens" if finish_reason == "length" else "end_turn")
+        write_anthropic_open_stream_stop(handler, {"stop_reason": stop_reason, "usage": {"output_tokens": output_tokens or max(1, len(text_so_far) // 4)}})
+    finally:
+        try:
+            resp.close()
+        except Exception:
+            pass
+
+
+def upstream_http_error_message(exc: urllib.error.HTTPError, raw: str | None = None) -> str:
     if raw is None:
         raw = exc.read().decode("utf-8", errors="ignore")
     msg = raw.strip() or str(exc)
@@ -4636,7 +4930,7 @@ def retryable_timeout_exception(exc: BaseException) -> bool:
     return "timed out" in text or "timeout" in text
 
 
-def post_json_with_rate_retry(
+def post_json_with_rate_retry(
     url: str,
     req_body: Any,
     headers: dict[str, str],
@@ -4698,50 +4992,129 @@ def post_json_with_rate_retry(
                 continue
             write_router_activity("error", provider, model, error=type(exc).__name__, tokens=token_estimate, bytes=byte_estimate)
             raise RuntimeError(f"{type(exc).__name__}: {exc}") from exc
-    raise RuntimeError("upstream request failed")
-
-
-def forward_openai_compatible_chat(handler: BaseHTTPRequestHandler, provider: str, pcfg: dict[str, Any], body: dict[str, Any]) -> None:
-    _update_tool_schema_registry(body.get("tools"))
-    model = resolve_requested_model(provider, pcfg, body.get("model"))
-    if provider == "nvidia-hosted":
-        model = ncp_model_id_for_nvidia_hosted(model)
-    req_body = openai_compatible_chat_request(provider, model, body, pcfg, stream=False)
-    url = join_url(provider_upstream_request_base(provider, pcfg), "/chat/completions")
-    waited, rpm_used, rpm_limit = apply_router_rate_limit(provider, pcfg, model)
-    stream = bool(body.get("stream", True))
-    notice = rate_limit_notice(waited, rpm_used, rpm_limit, bool(pcfg.get("rate_limit_status", True)))
-    if stream:
-        write_anthropic_open_stream_start(handler, model)
-        index = 0
-        if notice:
-            index = write_anthropic_stream_blocks(handler, [{"type": "text", "text": notice}], index)
-        try:
+    raise RuntimeError("upstream request failed")
+
+
+def open_openai_stream_with_rate_retry(
+    url: str,
+    req_body: Any,
+    headers: dict[str, str],
+    timeout: float,
+    provider: str,
+    pcfg: dict[str, Any],
+    model: str,
+    retry_notice: Callable[[str], None] | None = None,
+) -> Any:
+    gateway_retries = positive_int(pcfg.get("gateway_retries")) or 2
+    max_attempts = max(1, gateway_retries + 1)
+    token_estimate = estimate_tokens(req_body)
+    byte_estimate = len(json.dumps(req_body, ensure_ascii=False).encode("utf-8"))
+    data_bytes = json.dumps(req_body).encode("utf-8")
+    for attempt in range(max_attempts):
+        try:
+            write_router_activity(
+                "request",
+                provider,
+                model,
+                attempt=attempt + 1,
+                total=max_attempts,
+                tokens=token_estimate,
+                bytes=byte_estimate,
+                timeout=timeout,
+                stream=True,
+            )
+            router_log("INFO", f"upstream_stream_request provider={provider} model={model} attempt={attempt + 1}/{max_attempts} tokens={token_estimate} bytes={byte_estimate} timeout={timeout}")
+            req = urllib.request.Request(url, data=data_bytes, headers=headers, method="POST")
+            resp = urllib.request.urlopen(req, timeout=timeout)
+            learn_router_rate_limit_headers(provider, pcfg, model, resp.headers)
+            return resp
+        except urllib.error.HTTPError as exc:
+            raw = exc.read().decode("utf-8", errors="ignore")
+            learn_router_rate_limit_headers(provider, pcfg, model, exc.headers)
+            if exc.code == 429 and attempt == 0:
+                wait = register_router_rate_limit_backoff(provider, pcfg, model, exc.headers.get("Retry-After"))
+                time.sleep(wait)
+                continue
+            if exc.code in UPSTREAM_RETRY_HTTP_CODES and attempt + 1 < max_attempts:
+                retry_no = attempt + 1
+                write_router_activity("retry", provider, model, attempt=retry_no, total=gateway_retries, code=exc.code, tokens=token_estimate, bytes=byte_estimate, stream=True)
+                router_log("WARN", f"upstream_stream_retry provider={provider} model={model} attempt={retry_no}/{gateway_retries} code={exc.code} tokens={token_estimate} bytes={byte_estimate}")
+                if retry_notice:
+                    retry_notice(upstream_retry_message(retry_no, gateway_retries))
+                time.sleep(upstream_retry_wait_seconds(retry_no))
+                continue
+            write_router_activity("error", provider, model, code=exc.code, tokens=token_estimate, bytes=byte_estimate, stream=True)
+            raise RuntimeError(upstream_http_error_message(exc, raw)) from exc
+        except (TimeoutError, urllib.error.URLError) as exc:
+            if retryable_timeout_exception(exc) and attempt + 1 < max_attempts:
+                retry_no = attempt + 1
+                write_router_activity("retry", provider, model, attempt=retry_no, total=gateway_retries, error=type(exc).__name__, tokens=token_estimate, bytes=byte_estimate, stream=True)
+                router_log("WARN", f"upstream_stream_retry provider={provider} model={model} attempt={retry_no}/{gateway_retries} error={type(exc).__name__} tokens={token_estimate} bytes={byte_estimate}")
+                if retry_notice:
+                    retry_notice(upstream_retry_message(retry_no, gateway_retries))
+                time.sleep(upstream_retry_wait_seconds(retry_no))
+                continue
+            write_router_activity("error", provider, model, error=type(exc).__name__, tokens=token_estimate, bytes=byte_estimate, stream=True)
+            raise RuntimeError(f"{type(exc).__name__}: {exc}") from exc
+    raise RuntimeError("upstream stream request failed")
+
+
+def forward_openai_compatible_chat(handler: BaseHTTPRequestHandler, provider: str, pcfg: dict[str, Any], body: dict[str, Any]) -> None:
+    _update_tool_schema_registry(body.get("tools"))
+    model = resolve_requested_model(provider, pcfg, body.get("model"))
+    if provider == "nvidia-hosted":
+        model = ncp_model_id_for_nvidia_hosted(model)
+    url = join_url(provider_upstream_request_base(provider, pcfg), "/chat/completions")
+    waited, rpm_used, rpm_limit = apply_router_rate_limit(provider, pcfg, model)
+    stream_enabled = bool(pcfg.get("stream_enabled", True))
+    stream = True if provider == "nvidia-hosted" else bool(body.get("stream", stream_enabled)) and stream_enabled
+    notice = rate_limit_notice(waited, rpm_used, rpm_limit, bool(pcfg.get("rate_limit_status", True)))
+    if stream:
+        req_body = openai_compatible_chat_request(provider, model, body, pcfg, stream=True)
+        write_anthropic_open_stream_start(handler, model)
+        index = 0
+        if notice:
+            index = write_anthropic_stream_blocks(handler, [{"type": "text", "text": notice}], index)
+        try:
             def emit_retry_notice(text: str) -> None:
                 nonlocal index
                 index = write_anthropic_stream_blocks(handler, [{"type": "text", "text": text + "\n"}], index)
 
-            data = post_json_with_rate_retry(
-                url,
-                req_body,
-                provider_headers(provider, pcfg),
-                provider_request_timeout_seconds(pcfg),
-                provider,
+            resp = open_openai_stream_with_rate_retry(
+                url,
+                req_body,
+                provider_headers(provider, pcfg),
+                provider_request_timeout_seconds(pcfg),
+                provider,
                 pcfg,
-                model,
-                emit_retry_notice,
-            )
-        except RuntimeError as exc:
-            msg = str(exc)
-            write_anthropic_stream_blocks(handler, [{"type": "text", "text": f"Upstream error: {msg}"}], index)
-            write_anthropic_open_stream_stop(handler)
-            return
-        message = openai_chat_to_anthropic(data, model, source_body=body)
-        write_anthropic_stream_blocks(handler, list(message.get("content") or []), index)
-        write_anthropic_open_stream_stop(handler, message)
-        return
-    try:
-        data = post_json_with_rate_retry(
+                model,
+                emit_retry_notice,
+            )
+            stream_openai_chat_to_anthropic_sse(
+                handler,
+                resp,
+                model,
+                provider,
+                source_body=body,
+                start_index=index,
+                word_chunking=bool(pcfg.get("stream_word_chunking", False)),
+            )
+            write_router_activity("success", provider, model, tokens=estimate_tokens(req_body), bytes=len(json.dumps(req_body, ensure_ascii=False).encode("utf-8")), stream=True)
+        except RuntimeError as exc:
+            msg = str(exc)
+            write_anthropic_stream_blocks(handler, [{"type": "text", "text": f"Upstream error: {msg}"}], index)
+            write_anthropic_open_stream_stop(handler)
+            return
+        except Exception as exc:
+            msg = f"{type(exc).__name__}: {exc}"
+            write_router_activity("error", provider, model, error=type(exc).__name__, stream=True)
+            write_anthropic_stream_blocks(handler, [{"type": "text", "text": f"Upstream error: {msg}"}], index)
+            write_anthropic_open_stream_stop(handler)
+            return
+        return
+    req_body = openai_compatible_chat_request(provider, model, body, pcfg, stream=False)
+    try:
+        data = post_json_with_rate_retry(
             url,
             req_body,
             provider_headers(provider, pcfg),

package/docs/README.ja.md

@@ -47,7 +47,7 @@ vLLM、NVIDIA hosted、self-hosted NIM を選択し、通常の Claude Code 引
 
 Credits: One Ciel LLC
 
-現在のバージョン: `0.1.35`
+現在のバージョン: `0.1.37`
 
 ## 作られた理由
 
@@ -351,6 +351,22 @@ Windows/Linux 管理、クリーンアップスクリプト、定期的なセキ
 
 ## 変更履歴
 
+### 0.1.37
+
+- **Pseudo tool-call recovery**: NVIDIA/OpenAI-compatible stream 経路で
+  `<|tool_calls_section_begin|>...` pseudo tool-call テキストを画面に出さず、
+  可能な場合は Claude `tool_use` ブロックへ復元します。
+- **Streaming defaults**: provider streaming の既定値は on です。NVIDIA hosted
+  は安定性のため upstream streaming 経路に固定されます。
+
+### 0.1.36
+
+- **NVIDIA upstream streaming**: NVIDIA hosted router 呼び出しは upstream にも
+  `stream=true` を使用します。長い応答を完全な non-streaming completion まで
+  待たず、chunk として流せます。
+- **Stream retry diagnostics**: streaming NVIDIA 呼び出しでも statusline 用の
+  retry/request size activity 状態を維持します。
+
 ### 0.1.35
 
 - **NVIDIA router context guard**: NVIDIA hosted の router context 既定値を 32K

package/docs/README.ko.md

@@ -47,7 +47,7 @@ NVIDIA hosted, self-hosted NIM을 선택하고, Claude Code의 일반 인자는
 
 Credits: One Ciel LLC
 
-현재 버전: `0.1.35`
+현재 버전: `0.1.37`
 
 ## 왜 만들었나
 
@@ -351,6 +351,22 @@ Windows 이벤트 로그 리뷰, 바이러스/랜섬웨어 침입 시도 정리,
 
 ## 변경 이력
 
+### 0.1.37
+
+- **Pseudo tool-call recovery**: NVIDIA/OpenAI-compatible stream 경로에서
+  `<|tool_calls_section_begin|>...` pseudo tool-call 텍스트를 화면에 출력하지
+  않고 가능한 경우 Claude `tool_use` 블록으로 복구합니다.
+- **Streaming defaults**: provider streaming 기본값은 on이며, NVIDIA hosted는
+  안정성을 위해 upstream streaming 경로로 고정됩니다.
+
+### 0.1.36
+
+- **NVIDIA upstream streaming**: NVIDIA hosted router 호출은 이제 upstream에도
+  `stream=true`를 사용합니다. 긴 응답을 전체 완료까지 기다리지 않고 chunk로
+  흘려보내 timeout 가능성을 낮춥니다.
+- **Stream retry diagnostics**: streaming NVIDIA 호출도 statusline에서 쓰는
+  retry/request size activity 상태를 유지합니다.
+
 ### 0.1.35
 
 - **NVIDIA router context guard**: NVIDIA hosted의 router context 기본값을 32K로

package/docs/README.zh.md

@@ -47,7 +47,7 @@ NIM，并把普通 Claude Code 参数原样传递。
 
 Credits: One Ciel LLC
 
-当前版本: `0.1.35`
+当前版本: `0.1.37`
 
 ## 为什么存在
 
@@ -337,6 +337,21 @@ Hermes 格式模型或部分较旧的 Qwen tool template。
 
 ## 更新日志
 
+### 0.1.37
+
+- **Pseudo tool-call recovery**：NVIDIA/OpenAI-compatible stream 路径现在会
+  隐藏 `<|tool_calls_section_begin|>...` pseudo tool-call 文本，并尽可能恢复为
+  Claude `tool_use` block。
+- **Streaming defaults**：provider streaming 默认开启；NVIDIA hosted 为了稳定性
+  固定使用 upstream streaming 路径。
+
+### 0.1.36
+
+- **NVIDIA upstream streaming**：NVIDIA hosted router 调用现在也会向 upstream
+  使用 `stream=true`，长响应可以按 chunk 流出，不再等待完整的非流式 completion。
+- **Stream retry diagnostics**：streaming NVIDIA 调用也保留 statusline 使用的
+  retry/request size activity 状态。
+
 ### 0.1.35
 
 - **NVIDIA router context guard**：NVIDIA hosted 的 router context 默认值改为

package/docs/manual.md

@@ -10,7 +10,7 @@ Code starts, while passing normal Claude Code arguments through unchanged.
 
 Credits: One Ciel LLC
 
-Current version: `0.1.35`
+Current version: `0.1.37`
 
 ## Install
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@oneciel-ai/claude-any",
-  "version": "0.1.35",
+  "version": "0.1.37",
   "description": "Claude Code provider selector for Anthropic, Ollama, Ollama Cloud, vLLM, NVIDIA hosted, and self-hosted NIM.",
   "license": "MIT",
   "author": "One Ciel LLC",